Token prices have fallen for three straight years. AI bills have not. According to Gartner's August 2026 forecast, inference costs per agentic workflow will rise more than fivefold through 2028, even as the price of the underlying models keeps dropping.
Gartner calls this the Inference Paradox. For any leader who approved an AI agent budget on the assumption that "AI is getting cheaper", it is the most important cost concept of the next two years.
This guide defines the term, explains the mechanics at board level, and gives you a four-lever framework for keeping agent costs tied to business value.
What Is the Inference Paradox?
The Inference Paradox is Gartner's name for a simple pattern: the price of each AI token keeps falling, but the total cost of getting work done with AI agents keeps rising. Cheaper models make more ambitious agents affordable, and those agents consume far more computation per task than a chatbot ever did.
Inference is the computation a model performs every time it produces an answer. You pay for it in tokens, the units of text a model reads and writes.
A chatbot runs one round of inference per question. An AI agent plans, calls tools, checks its own output and tries again. Each of those steps is another round of inference.
The paradox is that falling unit prices encourage organisations to build agents that do more steps, so total consumption grows faster than prices fall. Gartner's earlier 2026 projection that large-model token costs will fall by more than 90% by 2030 does not contradict its fivefold forecast. Both are true at once.
Why Do AI Agent Costs Rise When Token Prices Fall?
Agent costs rise because the unit of consumption has changed. Leaders used to buy answers; they now buy completed workflows. A single workflow can involve dozens of reasoning steps, tool calls and retries, and every step re-reads a growing context. Volume growth outruns price cuts.
Three mechanics drive the increase.
--- Reasoning steps multiply. Reasoning models "think" before answering, generating internal tokens you pay for but never see.
--- Context compounds. An agent working through a ten-step task often re-sends its full history at each step, so step ten can cost many times more than step one.
--- Capability invites ambition. As The National CIO Review summarised Gartner's view, efficiency gains get reinvested into products that do more sophisticated work and consume more tokens along the way.
The result: the token price becomes a poor proxy for what AI actually costs your organisation.
How Much More Does an AI Agent Cost Than a Chatbot?
Gartner estimates that an agentic reasoning model costs a provider at least five times more per interaction than a basic chatbot exchange. Gartner analyst Will Sommer has put demanding workflows, with more reasoning steps and tool calls, at between five and thirty times the cost of a basic chatbot interaction.
That range matters more than any single figure. The same "invoice-matching agent" can sit at the bottom or the top of it depending on how it is designed.
An agent that uses a frontier reasoning model for every step, re-reads every document on every pass and retries without limit will land near the top. An agent that routes routine steps to a small model and caps its retries can land near the bottom.
One caveat for your finance team: Gartner's figure measures provider-side cost per workflow. It is not a direct forecast of your total AI spend or your vendor invoice, which depend on contracts and pricing models. Treat it as a directional warning, not a budget line.
Why Does the Inference Paradox Matter for Hong Kong Enterprises Now?
Hong Kong enterprises are moving from AI pilots to production agents in 2026, which is exactly when the paradox starts to bite. Pilot budgets were sized on chatbot-era usage. Production agents run continuously, at scale, across departments, and the cost curve changes shape the moment they go live.
Three local conditions sharpen the risk.
--- Adoption is already broad. The Privacy Commissioner's 2026 compliance checks found that 95% of the 60 organisations reviewed use AI in daily operations, up from 80% in 2025.
--- Value is still unproven. Gartner's October 2026 CIO research found only 13% of CIOs report significant value from AI so far. Rising run costs against flat value is how programmes get cancelled.
--- Agents are multiplying unmanaged. When nobody keeps an inventory of running agents, nobody owns their bills. We covered this risk in our guide to AI agent sprawl.
How Should Enterprises Measure AI Agent Costs?
Measure cost per completed task, not cost per token or cost per seat. Cost per completed task divides total inference, tooling and human-review spend by the number of tasks the agent finished correctly. It is the only number that can be compared directly against the value of the work.
Token dashboards tell engineers what happened. They do not tell a COO whether the agent is worth running.
A useful cost-per-task metric includes four components:
--- Model inference across every step, including retries.
--- Tool and data costs, such as search, database queries and API calls.
--- Human review time for outputs that need checking.
--- Failure cost: tasks the agent abandoned or got wrong, which a person then redid.
A simple worked example shows why this matters. If an agent costs HK$9,000 a month to run, needs HK$6,000 of staff review time and completes 3,000 tasks correctly, its cost per completed task is HK$5. If a third of its outputs later need redoing, the true figure rises sharply once the rework hours are added back.
That rework line is the one most dashboards leave out, and it is often the one that decides whether an agent pays for itself.
Set this against the value of the task, such as the fully loaded cost of a staff member doing it. If the ratio does not work, no token discount will rescue it. Our explainer on AI FinOps covers how finance teams are building this discipline.
What Is the Four-Lever Framework for Controlling Agent Costs?
The four levers are task tiering, step budgets, context discipline and a value threshold. Together they match the amount of intelligence an agent consumes to the value of the task. Gartner's own guidance points the same way: inference tiering, routing and orchestration rather than defaulting to one expensive model for everything.
Lever 1: Task tiering. Classify each workflow step as routine, judgement or high-stakes. Route routine steps to small, cheap models and reserve frontier reasoning models for the steps that genuinely need them. Gartner does not expect any single model to serve every workload economically.
Lever 2: Step budgets. Cap the number of reasoning steps, tool calls and retries per task. An agent that cannot finish within budget should escalate to a person, not keep spending.
Lever 3: Context discipline. Trim what the agent re-reads at each step, cache repeated material and summarise long histories. Context is often the largest hidden cost driver.
Lever 4: Value threshold. Agree in advance the maximum cost per completed task the business will accept. Above it, the workflow is redesigned or stopped.
The levers work as a set. Tiering without step budgets still allows runaway loops; budgets without a value threshold still allow unprofitable workflows.
What Does Agent Cost Control Look Like in Practice?
In practice, cost control is a design decision made before launch, not a finance exercise after the first invoice. A typical case is a Hong Kong logistics group whose customs-documentation agent works well in a pilot but costs several times its budget in production because every step uses the most capable model.
Consider an illustrative scenario. A 300-person logistics company deploys an agent to check shipping documents against customs requirements. In the pilot, it handles 200 files a month and the bill is trivial.
In production it handles 8,000 files a month. Each file triggers extraction, cross-checking against two databases, a compliance review and a summary. Every step runs on a frontier reasoning model, and failed checks retry without limit.
Applying the four levers changes the economics. Extraction and formatting move to a small model. Retries are capped at two before a human takes over. The agent stops re-reading the full document at every step. The company sets a ceiling on cost per cleared file, linked to the cost of a documentation clerk's time.
Accuracy on the hard cases stays the same, because the frontier model still handles the compliance judgement. Only the routine work gets cheaper.
What Mistakes Do Enterprises Make With AI Agent Budgets?
The most common mistake is budgeting agents like software seats: a fixed annual number set from pilot usage. Agents behave more like utilities, where consumption scales with volume, complexity and design choices. Budgets set without usage ceilings, owners and review points tend to fail within the first two quarters of production.
Four pitfalls recur.
--- Extrapolating from the pilot. Pilot volume and production volume differ by orders of magnitude, and so does context length.
--- One model for everything. Gartner warns that defaulting to generic autonomous intelligence can produce costs orders of magnitude above an optimised design.
--- No owner per agent. If an agent's bill lands in a shared IT cost centre, nobody has an incentive to optimise it.
--- No stopping rule. Without an agreed threshold, unprofitable agents keep running because switching them off feels like admitting failure. Our guide to AI stopping rules sets out how to define one.
What Should Enterprise Leaders Do Next?
Start by listing every AI agent in production or planned for the next two quarters, then estimate cost per completed task for each. Apply the four levers to the three most expensive workflows first. Present the board with cost per task against value per task, not token prices.
The Inference Paradox is not a reason to slow down on agents. It is a reason to design them deliberately. Leaders who treat intelligence as a resource to be allocated, rather than a utility to be switched on, will get the capability gains without the budget shock.
The organisations that win in 2027 will not be the ones with the cheapest tokens. They will be the ones who know exactly what each completed task costs and why.
That is the kind of clarity a long-term partner should bring to the table. We understand AI. We understand you. With UD by your side, AI never feels cold.
Want to Know What Your AI Agents Really Cost?
Once you have the framework, the next step is mapping it to your own workflows. UD's enterprise team will walk you through every step, from AI readiness assessment and workflow cost modelling to model selection, deployment and performance tracking. Twenty-eight years of Hong Kong enterprise IT experience, alongside you the whole way.
Reviewed by the UD enterprise AI team. Sources checked on 27 October 2026.