Last week I argued that open models will absorb most of the money and compute the world spends on AI. A week later, open weights are even more central to the conversation. Recent releases have made the gap between capable and affordable harder to ignore, and a broad coalition of technology companies is now publicly arguing that open models matter for competition, security, and national sovereignty. But the models themselves are only the starting point. The more consequential shift is what teams can now do with them. They can increasingly turn generic weights into specialized intelligence they own.
Read every issue? Consider becoming a paid supporter 🙏
The Post-Training Stack Is Filling In
Two years ago, taking an open model and making it meaningfully better at your specific job was already becoming easy to do. Supervised fine-tuning, meaning training on labeled examples of good output, was always the accessible option. The newer layer is reinforcement fine-tuning, where a model practices a task and learns from whether it succeeded. More than 25 startups are now building around this idea, but the algorithm is rarely the whole product. They are assembling the less glamorous machinery around it: environments where models can practice, graders and verifiers that decide whether the work was done correctly, tools for generating and curating data, and systems for evaluation, deployment, and monitoring.
Platforms are also starting to automate more of the engineering loop. A team can describe a task, establish a baseline, identify failure patterns, build evaluations and test data, train a model against those failures, and repeat. This still requires judgment, and reinforcement learning is not suddenly a push-button exercise. But what used to be a loose collection of research techniques is becoming a recognizable engineering workflow that more companies can adopt.
The demand side explains why this is happening now. Teams that put generic models into production tend to hit the same three walls. Prompt adjustments turn into whack-a-mole as each fix creates another failure. Costs become difficult to ignore once usage reaches real scale. Then a provider changes a model, retires it, raises the price, or starts competing in the same market as its customer. At that point, renting a general-purpose model stops looking like the obvious long-term architecture.
Specialization is attractive because most enterprise AI work is narrower than the frontier labs’ ambitions. It is classification, extraction, forecasting, tool selection, and the execution of repeatable workflows. In examples described to me, specialized models improved performance by 10 to 30 percentage points on well-defined tasks. In one case, a classifier with only a few hundred million parameters ran on a device and matched a model roughly ten times larger.
From Customizing Models to Creating Them
​​A smaller group is going further and pretraining models from scratch. Datadog is my favorite example because the logic is so plain. They sit on an absurd volume of time-stamped operational data, so they trained a time-series foundation model on it. Two details stuck with me. The model performs better when they exclude public time-series datasets entirely and train only on their own data, which is about as direct a statement of where the value sits as you will find. It is also cheaper to run than the classical statistical pipeline it replaced, once you count the tuning those older methods need on every query. Their proprietary data stopped being an input to the product and became part of the model itself. They are now aiming at something more ambitious, a model of their customers’ distributed systems built from metrics, logs, traces, topology, code, and events, meant to predict what a system will do next rather than report what it already did.
As the lineup for the upcoming Ray Summit hints, plenty of other companies are heading down the same road. A payments company is pretraining on transaction data. A design platform tokenized 900 terabytes across 18 datasets to train a model that generates geometry from text, images, and partial CAD input. A streaming service runs training, fine-tuning, evaluation, and serving on the same shared clusters. None of these are research projects, and two changes explain why. Domain models stay small. The 3D model above runs between 513 million and 1.5 billion parameters and Datadog’s largest is 2.5 billion, which is a rented cluster and a strong team, not a national research budget. And the platform underneath has converged on what I have been calling the PARK stack, meaning PyTorch, Ray, and Kubernetes handling data prep, training, post-training, evaluation, and serving in one place.
None of which makes pretraining easy or sensible for the average company. It becomes plausible when you have distinctive data, a workload worth repeating, and the team to keep the model running over time. The hard part is not the training run. It is knowing what data to feed it and in what mix, which is still closer to craft than process. Tooling will chip away at that, and as it does the question shifts from “could a company like ours train a model?” to “which parts of our business are worth one?”
The Moat Moves Into the Model
Put the two developments together and you get the shift toward owned specialization. Post-training is the on-ramp and pretraining is the far end. Most companies will operate somewhere between them, but the distance from “we have a use case and some data” to “we own a model that fits it” keeps getting shorter.
That is the deeper challenge for OpenAI and Anthropic. It is not simply that another open model might match them on a leaderboard. It is that their best customers can increasingly turn proprietary data, workflows, tools, and production feedback into intelligence that compounds inside their own organizations. Once enough companies build that flywheel, the big labs stop competing only against one another. They start competing against their own customers’ data. That is the headwind I do not think they have priced into their IPO story.
If you want to see this up close, Ray Summit has entire tracks on foundation model training, multimodal data curation, and reinforcement learning.
