Every enterprise AI conversation eventually becomes a cost conversation. The question is whether it happens at design time or at renewal.
Most enterprise AI conversations still treat cost as a later problem. Build the capability, prove the value, optimise the spend afterwards. That is a reasonable sequence for software. It is the wrong sequence for AI.
I spent fifteen years in infrastructure. In that world the expensive decision was a capital one — you sized the estate, you signed for it, and after that the marginal cost of using what you had bought rounded to nothing. Fifteen years of instinct, all of it built on the assumption that capacity is something you own.
Almost none of those instincts survive contact with inference — and what strikes me is how rarely that comes up. The questions I am asked are capability questions: what can it do, how fast can we deploy, which model is ahead this quarter. Far fewer are about what one unit of that capability costs once everybody is using it. That is not a knowledge gap. It is a sequencing habit carried over from an era when infrastructure cost was fixed at the moment you bought it.
The cost did not disappear. It moved.
For most of the last two decades, the expensive decision was the data centre. Capital, fixed, amortised over years. You bought capacity and then used as much of it as you liked — the marginal cost of one more query rounded to nothing.
Inference inverts that. The expensive decision is now the token. Every prompt, every retrieval, every agent hop is a priced event. Capacity is no longer something you buy once; it is something you spend, continuously, in proportion to how much people use the thing you built.
That single change moves cost out of procurement and into architecture.
The demo cost nothing because the demo was small.
A pilot is one user, one clean question, one call. Production is not that shape.
A retrieval step is a call. A reasoning chain is several. An agent that checks its own work is more again. One request that fans out into three agent hops and a retrieval is not a single priced event — it is five. Then multiply by every user, every session, every day.
None of this shows up in the pilot, because the pilot never ran at the volume where it matters. Illustratively, not as measurement: if a workflow makes five priced calls instead of one, and ten thousand people use it daily instead of ten, you are not facing one multiplier or the other. You are facing both, multiplied together.
The only figure that means anything is your own baseline, measured at production shape. Anyone quoting you a cost per token without knowing how many calls your workflow makes is quoting the wrong number.
This is not an argument for cheaper models.
Nor is it an argument against ambition.
Defaulting to the cheapest model everywhere is the same mistake pointed the other way. You pay for it in accuracy, retries and human cleanup — which are also costs, just less legible ones, and they land on people rather than invoices.
The argument is narrower than that: placement is a design decision, and it has to be made at design time.
Hybrid and multi cloud, read correctly
Hybrid and multi cloud have been sold for a decade as a hedge — risk management, avoiding lock-in, a diplomatic compromise between competing preferences. Read that way, they are a compromise.
Read as an economic control system, they are something else entirely: the mechanism by which each workload sits where its unit economics and its data obligations both work. Latency-sensitive inference close to the user. Bulk training where capacity is cheapest. Regulated data where it is legally required to stay.
Not one answer for the estate. One answer per workload.
I work this through two frameworks I keep coming back to: a “Workload Sovereignty Matrix” for the placement decision itself — which workload sits where, judged on unit economics and data obligation together, rather than one at a time — and an “Intelligence Control Stack” for the layers that keep those placements holding once the estate is actually running. Each deserves its own edition rather than a paragraph here.
Sovereignty and economics turn out to be the same lever, pulled for different reasons.
Where the money actually goes
MIT’s The State of AI in Business 2025 found enterprise AI budgets skewing heavily toward sales and marketing — somewhere between half and seventy per cent — while back-office applications delivered clearer returns. The study’s methodology has been publicly contested, and it deserves that scrutiny; I cite it for direction, not precision.
Read it as an economics finding rather than a functional one. Budget follows visibility. Unit economics follow workload shape. When those two disagree, the visible programme usually wins the funding while the quieter one compounds value nobody is counting.
What the leader has to keep
AI keeps offering to make the cost question feel premature. The demo is impressive, the capability is real, and this quarter’s spend is small enough to look like a rounding error. Every incentive in the room says prove the value now and price it later.
But at token-level economics, the cost structure is not set during optimisation. It is set the moment someone decides how many calls a workflow makes, how much context it carries, and where it runs. By the time the invoice is legible enough to argue about, the architecture has already answered the question.
So the question that stays with you is not whether you can afford the pilot. It is: what does one unit of this cost at the scale we actually intend — and does that survive our P&L?
That is not a finance question to be delegated after the fact. It is a design constraint, set before the build.
Ambition is what gets a programme funded. Economics is what decides whether it is still running in three years.