On October 28, 2026, TokenPost reported that its token expenditure price index for AI inference stood at $0.94 per million tokens on October 27. That is 54.48% below the period high of $2.065 it hit on May 28. Four weeks before that, on September 30, TrendForce told the memory market that conventional DRAM contract prices would rise another 10 to 15% in the fourth quarter, and that enterprise SSDs were the one category where price growth would speed up. Two price curves are moving in opposite directions, and most finance teams are only watching one of them. When I talk to executives about AI total cost of ownership this autumn, they nearly all quote me the falling curve. The cost they will actually pay sits on the rising one.
My argument is simple and a little uncomfortable. The token has become the cheapest and most visible part of running AI, so it has become the decoy. The money has moved down the stack into memory, storage, unbundled cloud line items and reserved capacity. Those costs are rising because of the AI boom, and they land on invoices your AI budget owner never sees. A company can watch its model spend fall by half and still find its total cost of ownership going up.
Two curves, one budget
Start with the curve everyone likes. Per-token prices have been falling for three years, and in 2026 they fell hard. TokenPost's index lost more than half its value in five months. If you run a contact-center copilot or a document pipeline on a frontier API, your per-call cost almost certainly fell this year, and somebody in your company has put that in a slide.
Now the other curve. TrendForce's September 30 release says that "Conventional DRAM contract prices are projected to grow 10, 15% QoQ in 4Q26," with NAND Flash contract prices expected to rise 15 to 20%. It says server DRAM "is therefore expected to remain undersupplied in 4Q26." It also says "Enterprise SSD price growth is therefore expected to accelerate in 4Q26," with bit demand projected to grow more than 80% year over year in 2026. On October 20, TrendForce passed along a Commercial Times report that fourth-quarter DRAM contract prices could rise about 20% quarter on quarter, after increases of about 80% in the second quarter and about 40% in the third. A slowdown to 20% counts as the good news in that market.
Those numbers are not about AI accelerators alone. They describe the memory in the ordinary servers that run your ERP, your data warehouse, your logging stack and the vector database your retrieval system queries a few million times a day. The AI boom pulled memory supply toward high-bandwidth memory for GPUs, and conventional server memory got scarce. RocketDevs, a developer-hiring firm that keeps track of cloud bills, cited Bureau of Labor Statistics data showing US producer prices for computer storage device manufacturing up 28.1% between September 2025 and June 2026. Over the same period, it notes, semiconductor prices overall fell 1.6%. Chips in general got cheaper. Memory and storage did not.
Both curves come from the same demand. Labs and hyperscalers buy so much compute that serving tokens gets cheaper per unit, and the same buying spree drains the memory pool your non-AI systems draw from. You get the discount on one invoice and the surcharge on another.
Where AI total cost of ownership actually lives
A standard AI TCO model has four lines: licenses or API spend, integration, people, and governance. Vendor guides add "hidden costs" as a fifth line. Almost none of them have a line for the state that AI systems create and keep, and that state is where costs are now rising fastest.
Here is what that state looks like in practice. A chatbot from 2023 took a question and returned an answer. A 2026 agent reads a ticket, pulls six documents, calls three tools, writes a plan, checks it, retries a failed step and logs every move so a human can audit it later. Each of those steps leaves something behind. You get embeddings in a vector index, cached context, tool outputs, trace logs kept for compliance and evaluation datasets built from those traces. The model call is a moment. The data it leaves behind sits on disks and in memory for months, sometimes years.
So the cost of an agent follows the same direction as the TrendForce report. Context caches and vector indexes live in server memory. Traces and evaluation corpora live on enterprise SSDs, the category TrendForce expects to accelerate this quarter. When a vendor says inference got cheaper, it is pricing the moment. Your bill for the leftovers is set by a commodity market that just had its strongest year in a decade.
For illustration, suppose a support agent costs four cents in tokens per resolved ticket in May and two cents in October. Suppose it also writes a trace, refreshes embeddings and keeps a replay log, and the storage and memory share of the work comes to three cents in May. If that share rises by a third, the ticket now costs six cents instead of seven. The AI team reports a 50% cut in model cost. The CFO sees a 14% cut in cost per ticket. Each is right about its own number. Only one of them describes your margin.
The invoice got unbundled
The second place the cost goes is into line items that used to be free. Cloud providers raise prices quietly, and in 2026 the quiet method is unbundling.
RocketDevs documented three cases. From October 1, 2026, OVHcloud began charging separately for local storage and public IPv4 addresses on its B3, C3 and R3 instances, which used to include them. The hourly sticker price stayed the same. Total costs for identical setups rose between 1.4% and 21.9%. In June, Hetzner raised several plans and hit memory-heavy configurations hardest. Its CCX13 in Germany and Finland went from €15.99 to €42.99 a month for new orders and rescaling. On May 1, Google Cloud doubled North American peering egress from $0.04 to $0.08 per GiB.
None of these are AI price increases on paper, and none would show up in a dashboard tagged "LLM spend." All three hit workloads that AI systems make bigger. Retrieval moves data across networks. Agents spread into many small services. Memory-heavy instances host the caches that keep inference cheap. Unbundling punishes architectures with many parts, and agent systems have more parts than anything most companies have run before.
The hyperscalers told you on October 28
If the memory market looks too far from your own operations to matter, the earnings calls on October 28 should change your mind, because there the substrate cost was stated outright.
CNBC reported that Microsoft cut its full-year capex forecast to $170 billion from $175 billion. The reason was delays at its third-party GPU providers, which will push some capacity out until the first half of 2027, and lower demand had nothing to do with it. CNBC also reported that Microsoft's CFO expects capital spending to accelerate in the first half of 2027 because of higher component costs and delayed deliveries. Microsoft's revenue held up. According to 24/7 Wall St., Azure grew 48% against a 44% consensus, and commercial remaining performance obligation, the contracted backlog, rose 73% to $677 billion. Amy Hood told investors that "each dollar of CapEx has the potential to support more revenue and higher returns over time."
Alphabet's quarter was blunter. 24/7 Wall St. reported capital spending of $51.3 billion, up from $44.9 billion the quarter before. Free cash flow was negative $9.1 billion, the second negative quarter in a row, and the company raised $28.7 billion in senior unsecured debt. Google Cloud revenue was $30.71 billion, up 103%. Alphabet's finance chief said outside computing power "came online only at the very end of Q3." The stock fell 11% the next day.
Put those facts next to the token index and a pattern shows. The companies selling you cheaper tokens are paying more for the components that make them, borrowing to pay for it, and waiting on delayed capacity. They can keep cutting per-token prices for a while, because the API is where they compete for share and they can fund the gap from the balance sheet. They can't keep it up forever. The $677 billion Microsoft backlog is customers locking in capacity at today's terms, which tells you how the largest buyers read the situation.
Why the curves split
The split has a structural cause, and you need to understand it to plan around it. Token prices are set at the top of the stack, where competition among labs, model distillation and better serving software keep pushing the unit cost of a response down. Memory prices are set at the bottom, by a few manufacturers who moved their fabs toward HBM because that is where the margin is. TrendForce's October 20 note gives one example: if NVIDIA's Rubin Ultra moves from 12-Hi to 8-Hi HBM stacks, the same DRAM die supply could in theory support 50% more GPUs. The industry is working hard to stretch memory across more accelerators. It is doing very little to stretch it across your database servers.
Efficiency at the top therefore doesn't reach the bottom. A smarter serving engine cuts the GPU-seconds per answer. It doesn't lower the contract price of the RDIMMs in the box that holds your embeddings. Labs pass software efficiency straight through to you as a discount. Hardware scarcity gets passed through too, more slowly and with a different label, on a line item someone else in your company approves.
The ownership math just changed
The obvious response is to own the hardware, and the case for that looks strong on paper. Cohere's July 15, 2026 piece on the total cost of AI ownership cites a Lenovo analysis in which owned H100 hardware generates a million tokens for "roughly $0.11" amortized, against "about $0.89" on an equivalent cloud instance and "around $2.00" on a comparable frontier API. Cohere says an 8-GPU server costs less to own than to rent at about four hours of use a day.
Cohere also gives the caveats, and they matter more now than in July. The owned figures "assume sustained high utilization." They are vendor-computed. The cloud side leaves out storage, egress and support, which are exactly the lines that went up this year. And a server's bill of materials is no longer fixed. If you buy servers in the fourth quarter, you are buying memory at a price TrendForce expects to rise again, and you are buying it once, at the top of a cycle, for a machine you will depreciate over years.
So self-hosting now carries a second bet on top of the utilization bet. Utilization decides whether the GPU pays for itself. The memory cycle decides what the rest of the box cost you. Suppose you buy at the peak and the memory market cools in 2027. You keep the high price in your depreciation schedule while your cloud-renting competitor gets the lower one. Suppose instead the shortage lasts. Then the owner wins, and the renter faces unbundled line items and rising instance prices. Neither choice is safe. What separates good decisions from bad ones is whether anyone at the table knows which bet they are placing.
What a real AI cost model looks like
None of this means you should slow down on AI. It means the cost model most companies use measures the wrong thing, and the error is growing. Here is what I would change, in rough order of how fast it pays back.
Price the outcome with its residue attached
Stop reporting cost per token or cost per call. Report cost per finished unit of work, such as a resolved ticket, a reconciled invoice or a reviewed contract, and include everything that unit leaves behind. That means the trace, the embeddings, the cached context and the evaluation sample. Divide storage and memory spend across those units the same way you divide token spend. Once you do, you will see that some of your "cheap" agents are expensive because of what they hoard.
The metric I push clients toward is bytes kept per outcome, tracked month by month next to tokens per outcome. Tokens per outcome tells you how efficiently the system thinks. Bytes kept per outcome tells you how much it hoards. In 2026 the second number has the stronger trend, and nobody has a dashboard for it.
Give agent traces an expiry date
Most teams keep every trace forever. They reason that the logs might be useful for evaluation, debugging or an audit, and storage used to be cheap enough that nobody pushed back. That reasoning no longer holds. Decide which traces are needed for regulators and keep those for the legal period. Decide which traces feed evaluation, sample them, and delete the rest on a schedule. Move cold data to cheaper storage tiers on purpose instead of by default. With enterprise SSD prices accelerating, a retention policy is one of the few cost controls that takes effect at once and asks nothing of the model.
Treat context length as a cost centre
Long context windows tempted everyone into stuffing them. Big prompts are cheap on the token line because cache reads are discounted, and they get expensive on the memory line, because someone has to hold that cache. If you self-host, you hold it in your own server memory. If you rent, the price of holding it is folded into a rate that your provider will adjust when its component costs force it to. Microsoft's CFO has already told you that is coming. Engineering teams should have to justify context size the way they justify database indexes, as a resource with an owner and a budget.
Contract for the bottom of the stack
TrendForce notes that some suppliers' server DRAM price increases may lag the market because of long-term agreement pricing. The memory buyers who signed long contracts are paying less than spot this quarter. Your company is probably not buying DRAM directly, but you are buying instances, storage and dedicated hosts whose prices follow it. Where you can, negotiate multi-year terms on the memory- and storage-heavy parts of your AI infrastructure, and spend less effort on token discounts. The token price is already falling without your help. The substrate price is the one that needs a contract.
Split the forecast in two
A single AI budget line hides the split this whole essay is about. Forecast the token line and the substrate line separately, with separate owners and separate assumptions. Assume the token line keeps falling, because the evidence says it will. Assume the substrate line follows memory and cloud pricing, which the evidence says is still rising. Then make the two owners present together. In most companies the person who celebrates the falling API bill has never met the person approving the storage renewal, and the AI total cost of ownership falls into the gap between them.
The decoy and the bill
Here is how it usually happens. A company launches agents in the spring and its model spend drops through the summer. Leadership hears that AI is getting cheaper and approves more agents. Each new agent writes more traces, holds more context and spreads across more small services. In the autumn the infrastructure renewals arrive, priced off a memory market that rose 80%, then 40%, then perhaps 20%, quarter on quarter. Nobody links the renewal to the agents, because the agents sit under "AI" and the renewal sits under "infrastructure." The CFO sees margin slipping and blames the wrong department.
Everything in that story is ordinary, and the facts behind it are public, from the TokenPost index to the TrendForce releases to the October 28 calls. The only thing missing in most companies is one model that puts the two curves in the same spreadsheet.
Design the cost before it designs you
You cannot fix this with a tool. No FinOps dashboard knows that the vector index in one account exists because of the agent in another, or that the jump in a storage renewal traces back to a retention default an engineer set in March. The connection lives in the architecture, so the fix has to be made there. That means deciding how agents store state and how long traces live. It means choosing which workloads you own and which you rent, knowing which memory-cycle bet each choice is. It means writing contracts that cover the bottom of the stack as well as the top. A company that makes those decisions deliberately can ride the falling token curve and stay off the rising one. A company that leaves them to defaults will keep reporting cheaper AI while its margins say otherwise.
That is the work we do at Agor AI Advisory. We map every cost an AI system creates, from the model call down to the storage and memory it leaves behind, and we redesign the system so the savings at the top don't get eaten at the bottom. We have read the earnings calls, the memory forecasts and the cloud invoices, and we build cost models that hold up when the next renewal arrives. The memory market will not wait for your next budget cycle, and your vendors have already told you their component costs are going up.
Sources
- TokenPost, AI Inference Price Measure Falls 1.9% Over Seven Days Through Oct. 27, October 28, 2026
- TrendForce, AI Server Demand Sustains Memory Contract Price Increases in 4Q26, September 30, 2026
- TrendForce, DRAM Prices Seen Rising 20% QoQ in 4Q26 Despite Slower Gains, October 20, 2026
- CNBC, AI buildout hits snag as Google and Microsoft put off raising capex forecasts, October 29, 2026
- 24/7 Wall St., Alphabet Sinks 11% as AI Spending Pushes Cash Flow Negative Again, October 29, 2026
- RocketDevs, Your cloud bill is rising while your AI bill falls, 2026
- Cohere, The Total Cost of AI Ownership, July 15, 2026