Gartner, Inc. on Monday published a prediction that underscores the hidden cost of the AI boom: inference costs per agentic workflow will increase more than fivefold through 2028, even as the price of foundation models keeps falling.
The apparent contradiction — cheaper models, more expensive bills — is what Gartner calls the "inference paradox." Falling token prices tempt companies to build increasingly complex agentic systems, and those systems consume so many more tokens that the savings are overwhelmed.
"Where a simple chatbot must read and interpret a query and quickly respond with a probabilistic reasonable answer, an AI agent must constantly reason, negotiate, and question itself," said Will Sommer, senior director analyst at Gartner.
According to the analyst firm, routing a task to an agentic reasoning model increases inference costs at least fivefold — and potentially by much more as tasks become more complex. The finding lands as Nvidia and other tech giants push inference and agentic AI as the next stage of the AI wave, and as some AI providers shift from flat-rate subscriptions to usage-based billing, leaving token-hungry workloads exposed to runaway bills.
To make the economics work, Gartner says organizations need either substantially greater returns than basic models deliver or far better optimization of inference, routing and orchestration — for example, assigning each task to the cheapest model capable of handling it.
The warning follows Gartner's earlier forecast this year that 40% of organizations would demote or decommission AI agents by the end of 2027 because of escalating costs, unclear business value or inadequate risk controls.
