Gradient Flow #38: Large Language Models, Infinite Laptop, Overhyping AI

Ben Lorica

5 years ago

This edition has 455 words which will take you about 3 minutes to read.

“Yesterday I was clever, so I wanted to change the world. Today I am wise, so I am changing myself.”- Rumi

Data Exchange podcast

Training and Sharing Large Language Models Connor Leahy, AI Researcher at Aleph Alpha GmbH, and founding member of EleutherAI, a collective of researchers and engineers building resources and models for researchers who work on natural language models. We discussed the state of natural language research, the rise of large language models, embeddings, and a new role that Connor and others refer to as “prompt engineer”.
Neural Models for Tabular Data Sercan Arik, Research Scientist at Google Cloud AI, and his collaborators recently published a paper on TabNet, a deep neural network architecture for tabular data. It uses sequential attention to select features, is explainable, and based on tests and actual deployments, TabNet outperforms or is on par with other models on classification and regression problems.

Featured Virtual Event

Our friend Paco Nathan is part of a strong roster of speakers at the upcoming Datanova For Data Scientists. He will moderate a session on the data lakehouse, an emerging data management paradigm:

Data & Machine Learning tools and infrastructure

Combine the development experience of a laptop with the scale of the cloud I compiled observations from keynotes at the recent Ray Summit. This post will be useful to anyone building AI applications or machine learning platforms.
Scaling Apache Pulsar to 10 PB/day Splunk reveals how it uses Pulsar
AugLy from Facebook AI Research A new open source library for data augmentation in computer vision and natural language models. Data augmentation tools expand your labeled training datasets and is particularly useful for model validation and improving model robustness.
Flat Data This clever open source project simplifies routine data acquisition and cleanup tasks. Since it runs on GitHub Actions, you won’t need specialized infrastructure to use it.
Facebook’s Prophet: a mostly negative review As Prophet’s co-creator, Sean Taylor notes, it is very challenging to build a forecasting tool that generalizes across many domains and problems. I’ve had decent luck using Prophet in combination with an optimization tool like Ray Tune.
How Edge Observability will enable companies to push compute to the edge New generation of tools combine distributed stream processing and federated machine learning capabilities for intelligent, dynamic, and automated data routing to optimal destinations.