Inference: the AI cost worth watching
Essay, 2026-07-01
Every model upgrade quietly changes your unit economics. The useful habit is knowing what a workflow costs to run, and how that shifts when the model does.
Inference is becoming one of the biggest constraints in AI. As adoption spreads across organisations and teams, the gap I keep running into isn't capability. It's cost. People are building genuinely good workflows: agents that plan, execute, and hand off to sub-agents, running on schedules, day after day. The part that often comes later is asking what those workflows actually cost to run at scale, and how that cost shifts every time the underlying model changes.
That question decides whether AI adoption in a business stays durable. You can design a workflow that produces genuinely useful output and still build a bottleneck if nobody understands what that output costs in tokens. Cost is easiest to manage when it's visible early, not when it surfaces as a surprise.
Why the economics keep shifting
The frontier labs are heading toward IPOs. Enterprise pricing is being restructured. The VC subsidy that's made frontier model access feel almost free is not going to hold forever. When it lifts, the relationship most people have with AI changes, and not just on the workflow or automation side, and not just in ROI terms. It changes at the level of inference itself: the tokens going in, the tokens coming out, and whether anyone in the business understands that trade.
Teams have got comfortable measuring AI in terms of what it produces. The next step is measuring what it costs to produce, and watching how that cost moves with every model release.
Defaulting to the newest model
The pattern I keep seeing is excitement about the newest, most powerful model the moment it ships, then setting it as the default for everything. Take Fable: it can produce genuinely strong output and act as a stand-in for skills a team doesn't have yet. That power comes at a cost. It's a slow model, hugely token-hungry, and if you set it as the default for everyday work, the bill climbs quietly without anyone meaning it to.
We're seeing the same pattern with Sonnet 5. It's a near-opus intelligence model, but it's slower than its predecessor, Sonnet 4.6, delegates tasks out to sub-agents in the background, and has reportedly been using significantly more tokens and costing more to run as a result. Each of those properties is easy to miss until you look for it, and each one multiplies the bill.
Choosing the right model for the job
Every time a model updates, the useful question isn't "is this the best one available." It's "which of the available models is the right one for this specific job." Once a workflow is built, test it against the full range: does a faster, lighter, flash-tier model produce comparable output for less? If the answer is no, can the thinking and planning stay on a stronger model while execution moves to something quicker and cheaper? Most workflows split cleanly along that line once you look for it.
Scheduled tasks are where this compounds fastest. A workflow that costs a few pence run once by a person becomes something else when it runs on a schedule, multiplied across every hour, every day. A short review of the bill, early and often, is what keeps a useful automation from becoming an expensive habit.
None of that works without a bit of token literacy. People need to know how input tokens work, how output tokens work, and how scheduled or agentic tasks magnify cost far faster than a single one-off prompt does. With every model release, it helps to see what the new pricing actually does to the business, rather than assuming better intelligence automatically means better value.
Inference is where AI economics gets real. The organisations paying attention to it now are the ones still running these workflows profitably in two years.
More field notes