Inference Is Eating Model Training

As language models mature, the real workload shifts from building them to running them—where every prompt turns into tokens, cost, and scale.

Transcript excerpt: I'm Kosh from Aerea. I'm the CEO and co-founder of Ornn. So we look at all these AI models that are large language models, so there's ChatGPT, Andropic, Gemini, and many others. The way that they actually run and the way that when you type into ChatGPT, hey, what is the weather today? And it comes out with a response. That process where the model actually generates the tokens is called inference, where you're able to actually use the model that's become trained. And as AI goes further and further, more and more of the world is going to move into inference-related tasks from training the actual model.