As major technology firms like Microsoft, Google, and Anthropic seek to monetize their massive investments in Large Language Models (LLMs), companies integrating these tools into their own products are facing a significant financial hurdle: the difficulty of predicting AI costs. Unlike traditional software, AI services rely on "tokens"—mathematical chunks of data used to process prompts and generate responses—which fluctuate in volume based on the complexity and nature of the task.

The unpredictability of these costs stems from the non-deterministic nature of AI outputs. Because identical prompts can yield different results and varying token usage, businesses often find it difficult to forecast expenses. This issue is compounded by the rise of "agentic AI," where multiple autonomous agents interact to perform tasks, leading to a surge in token consumption. Goldman Sachs projects that external token usage will grow 24-fold between 2026 and 2030, reaching 120 quadrillion tokens per month.

This volatility has already impacted major corporations. Reports indicate that Uber exhausted its annual AI coding budget in just a few months, while Microsoft has reportedly restricted its engineers' access to certain third-party coding tools to manage costs. Will Venters, an Associate Professor at the London School of Economics, notes that companies are struggling to manage these expenses because "it's a non-deterministic output, so it's a non-deterministic value."

Advertisement

To mitigate these risks, some organizations are attempting to use flat-fee personal accounts, though industry experts warn that major AI vendors will likely move to restrict this practice as they face shareholder pressure to increase profitability. Others are focusing on more precise prompting or carefully selecting which models to deploy for specific tasks. Despite these efforts, firms like Sumo Logic are still grappling with how to pass these variable costs on to customers without alienating them with unpredictable pricing.

"Nobody's really figured it out," said Bill Peterson, senior director of product marketing at Sumo Logic, regarding the search for a viable billing model. As the industry evolves, businesses remain caught between the potential value of AI-driven efficiency and the challenge of budgeting for a technology that changes its cost structure every few months.

Source: BBC News