AI’s real cost problem starts after the model call

Enterprise AI bills are rising faster than many companies expected, but model pricing is only part of the story. In this TechRadar Pro Perspectives piece, Laurynas Biveinis, co-founder and president of Cast AI, argues that the bigger cost drivers often sit behind the scenes: infrastructure, orchestration, retries, idle GPUs, oversized context windows, and poor routing decisions.

His warning is blunt: by the time teams see the invoice, the choices that shaped it were made weeks earlier in autoscaling policies, Kubernetes configurations, prompt design, and workflow architecture.

The three layers behind AI costs

Biveinis says enterprises often track only one part of a three-layer problem:

• Production: the infrastructure used to generate tokens, including GPU clusters, inference nodes, and scaling policies
• Consumption: how prompts, caching, retries, context windows, and model routing affect what gets spent
• Value: whether that token spend maps to business outcomes

According to the piece, most organizations are comfortable measuring value, but lack enough visibility into production and consumption to govern costs properly.

Recommended reading

UK data centre boom hits a water reality check

At FinOps X in San Diego, Biveinis points to a Goldman Sachs projection shown on the main stage: current global enterprise token consumption is about six quadrillion tokens, with a three-year projection of 120 quadrillion. He also notes the launch of the Tokenomics Foundation, a vendor-neutral body inside the Linux Foundation focused on the economics of AI token consumption.

Why routing and automation matter

One of the article’s central claims is that many companies overuse frontier models. Based on Biveinis' observations, only about 15% of software development tasks truly need those top-tier models, while the remaining 85% of routine work such as coding, summarization, classification, and retrieval can be handled by smaller and cheaper systems.

He argues that manual model selection does not scale. Instead, organizations need infrastructure that can automatically:

• identify the task
• route it to the right model tier
• evaluate output quality
• escalate when needed

That becomes more urgent as AI systems shift from assistants to autonomous agents. Once systems can plan, execute, evaluate, and iterate without a human at every step, the economics change sharply. A single request can fan out into dozens of model calls, each billed separately, while inefficiencies scale continuously in the background.

Biveinis' conclusion is that enterprises need more than dashboards. They need control at the infrastructure layer itself: autonomous management of GPU and inference workloads, continuous rightsizing, and the ability to move compute across providers when needed. In his view, the next phase of enterprise AI will be defined less by who can access models and more by who can deploy them efficiently.