What Startups Taught Me About the Next Layer of AI Infrastructure
A little while back I wrote about how teams use reinforcement learning (RL) to make agents reliable. Since then I keep bumping into startups where RL is not a research footnote or a feature buried in the stack. It is central to what they are building. I know of more than 25 at last count and still climbing, and in some cases RL is basically the product. It is still early, and much of the current tooling is aimed at frontier labs, neo labs, and sophisticated AI teams. Still, there is enough signal to think that some of the tools and lessons will eventually trickle down to the rest of us. At this point I treat my own RL timelines the way I treat horoscopes, entertaining but not binding. What I will say instead is that this particular wave has startups, products, and customers around it, which the previous periods mostly did not.
If you read often, consider becoming a paid supporter 🙏
The New Infrastructure for Learning by Doing
The common thread is that almost all of these startups are reacting to the same frustration. Frontier models are smart but unreliable the moment you ask them to do real, multi-step work. RL is the lever they pull to turn raw capability into dependable behavior, whether the target is a coding agent, a support bot, or a humanoid robot.
But none of them are really selling RL as a magic algorithm. They’re building the missing machinery around it, and that machinery has two hard parts. The first is giving a model somewhere to practice, which is why so many are in the business of simulated environments and training gyms. The second, and trickier, part is scoring. Did the agent finish the task, follow the rules, pick the right tool, avoid a shortcut that games the metric? A surprising amount of this market is really about graders, verifiers, and the tooling to catch an agent cheating its own reward.
From there the goals split. Some startups build RL environments for frontier labs. Some build post-training platforms for companies that want specialist models. Some focus on continuous learning from production traces. Others carry RL into robotics, industrial control, and logistics. The common thread is simple enough to state plainly. If you can define the task, create a safe place to practice, and score the result, RL has something to work with.
Where Learning by Doing Pays Off
The first commercial cluster is exactly where you would expect. Coding, software engineering, ML engineering, security, and infrastructure tasks show up again and again because they come with built-in ways to score progress. Code can compile or fail. Tests can pass or fail. A sandbox can reveal whether the agent’s action changed the system in the intended way. The feedback is still imperfect, but it is much cleaner than asking a human to grade every step from scratch.
The second big cluster is computer use and enterprise work. These are agents that operate browsers, spreadsheets, CRMs, ticketing systems, internal tools, finance software, documents, and knowledge bases. This is where the difference between a demo and a production agent becomes painfully obvious. A model that can explain a workflow is not the same as an agent that can execute it, recover from mistakes, and satisfy a company’s standards.
Then there’s a smaller but striking set of use cases that leave the screen entirely: robots learning to walk and grip, data-center cooling, freight routing. The wider pattern is that RL is showing up wherever teams can create a loop. There has to be a task, a place to attempt it, a way to observe what happened, and a score that is close enough to what the user actually cares about. That pattern applies to coding agents, enterprise research, support workflows, financial modeling, medical reasoning, industrial cooling, logistics, and robotics. The domains differ, but the recipe is surprisingly consistent.
When Does This Reach the Rest of Us
The question I keep coming back to is when this becomes usable by a broader set of AI teams. Right now, a lot of the tooling still assumes serious technical depth. You need to define environments, write rewards, inspect rollouts, catch reward hacking, and connect the training loop to real systems. That is not casual weekend work. But when this many startups are building environments, post-training platforms, verifiers, agent gyms, and RLOps tooling, it is hard not to expect some of the abstractions to become simpler.
In one sense, the trickle-down has already started. The startups helping companies post-train or customize foundation models are bringing pieces of RL tooling into enterprise settings today. That’s RL tooling reaching real businesses today, not in some hypothetical future. The next step is not that every AI team becomes an RL research lab. It is that more teams get practical tools for defining the work they care about, measuring success, and improving models against those signals over time.
