The cost of running an agentic workload is not a black and white matter. Because of how each of the different models compare in their behaviour on the same task, model X might use 10k tokens, while model Y might use 100k tokens. Additionally, the visible transcript is not the whole story. OpenAI, Anthropic and Google all hide or summarise the reasoning trail, and all three bill it as output tokens anyway. What you see the model write and what you pay for are already two different numbers.
Ultimately, what matters is how many total tokens the model spends, the speed it can output tokens, and how much each token costs. Completion comes first. Iterate until the model nails the task every time. Then the goal is faster, cheaper.
Available models on the market have a different consumption of tokens per task. They all behave varyingly, but most are able. And the intelligence benchmarks used to score the models are not always the right mark to gauge a model against. Because the workload you're applying the model to will be different than an agentic math test. In essence, desired model behavior and performance is something that is discovered and engineered rather than selected for.
Same score, three bills
The benchmark for DeepSWE shows the difference between two frontier models, and how they behave on "long-horizon engineering tasks". Datacurve wrote the 113 tasks from scratch across 91 open-source repositories, so no model has seen the solutions, and grades the committed code in a clean environment. When inspecting the leaderboard as of 3 September, we recognise a stark difference between Opus and Astra. On the same tests, Opus uses 3.9X the amount of tokens to complete the tasks (Opus spends 118k vs Astra 30k). When looking at the average cost of these two models specifically, Opus costs $11.84, compared to Astra at $6.52.
There is a third model tied at 74%. Gemini 3.8 Flash, released the day before the snapshot, spends 143k tokens per task, more than either, and costs $2.36. The most token-hungry model on the top edge is the cheapest by a factor of five.
1 GPT-6 Astra (xhigh). 74.1% pass@1, 30k output tokens, $6.52 per task, 29 steps.
2 Gemini 3.8 Flash (high). 73.8% pass@1, 143k output tokens, $2.36 per task, 166 steps.
3 Claude Opus 5 (max). 73.6% pass@1, 118k output tokens, $11.84 per task, 99 steps.
Clearly models behave very differently and have completely different cost profiles, even while completing the same tasks at identical scores.
1 GPT-6 Astra (xhigh). 74.1% pass@1, 30k output tokens, $6.52 per task, 29 steps.
2 Gemini 3.8 Flash (high). 73.8% pass@1, 143k output tokens, $2.36 per task, 166 steps.
3 Claude Opus 5 (max). 73.6% pass@1, 118k output tokens, $11.84 per task, 99 steps.
Why is that?
Token efficiency. Partly.
Output tokens are a minority of the bill. At list prices, Opus's 118k output tokens cost $2.95 of its $11.84. Astra's 30k cost $1.50 of $6.52. Gemini's 143k cost $0.54 of $2.36. Roughly three quarters of every bill is input, the context re-read on every step of the agent loop. Astra takes 29 steps to Opus's 99 and Gemini's 166. Steps times context is the lever. Those are estimates, because Datacurve has not published its cost method and its cache handling has been questioned, but the split holds at any plausible cache treatment.
The compute a provider chooses not to bill
The model API hides the reasoning chain from view, and bills it anyway. So the compute that can be hidden from the invoice is the compute the provider chooses not to bill. And there is a possibility worth considering here: What if a model provider deliberately chose to take on some of the cost of reasoning? Inference itself is very profitable, which makes it possible for the provider to subsidize some of the cost. When reviewing the larger model performance game, such a choice becomes rational.
SemiAnalysis wrote in December 2024 that "Gross margins on frontier models are north of 70%, but on trailing models with open source competition, margins are below 20%." In January 2026 they put frontier inference gross margins north of 60%, and in June they estimated Opus 4.8 at 85% or more. By August: "Serving inference tokens is unbelievably profitable for the frontier model companies." These are estimates of the margin on serving paid tokens. The reported company numbers are lower. The Information reported in January that Anthropic projected a 40% gross margin on its paid business for 2025 and OpenAI around 46% across all users, free included. Both can be true at once.
As inference gets cheaper, providers gain increasing freedom to hide computation cost without exposing that computation as the product's unit of value.
The reduced cost of inference tokens is driven by much higher tokens per second per GPU, better batching, caching, and newer accelerators. Epoch AI measured the price of reaching a fixed benchmark score falling between 9x and 900x per year. SemiAnalysis's InferenceX measures a rack of GB300s at up to 100x the realized throughput of a strong H100 setup.
That leaves considerable room between the cost of producing intelligence and the price charged for it. A provider does not necessarily have to return every efficiency gain to the user as lower prices. It could instead spend some of that margin on more inference.
Imagine an API call currently costs the provider $0.30 to serve and produces $1.00 in revenue. That is a 70% gross margin.
Now suppose the provider allows the model to reason longer, verify its work, consult an "advisor model" and retry internally. The underlying inference cost rises from $0.30 to $0.45, while the user-facing price remains $1.00.
The provider has deliberately reduced its gross margin from 70% to 55%.
But it has bought something with that margin: a better model.
If the resulting system completes more tasks, performs better on agent benchmarks, appears cheaper per successful task and attracts more developer workloads, sacrificing some margin on the individual inference may be economically rational.
This becomes a game-theoretic decision rather than a simple cost-plus pricing decision.
Game-theoretic means the payoff of a choice depends on what the other players choose. Cost-plus pricing looks inward: cost, plus a margin. A game-theoretic price looks across the table: what a rival will charge, what a developer will switch for, what a benchmark position is worth next quarter. Game theory is the study of how agents choose when each outcome depends on everyone's choices.
OpenAI, Anthropic and other frontier providers are not necessarily optimizing the profitability of a single token. They are competing for workloads, developers, benchmark position and ultimately the economic layer through which machine intelligence is purchased.
In that game, there is no reason to assume that every unit of computation performed internally must be reflected proportionally in the price visible to the user. A provider with sufficiently high inference margins could effectively subsidize reasoning.
There is no evidence here that OpenAI is currently doing this at material scale. It is a thought experiment. But the economics make the strategy possible.
The exit from tokens
This gets considerably more interesting as pricing moves away from tokens. OpenAI's CFO, Sarah Friar, wrote in January: "As intelligence moves into scientific research, drug discovery, energy systems, and financial modeling, new economic models will emerge. Licensing, IP-based agreements, and outcome-based pricing will share in the value created. That is how the internet evolved. Intelligence will follow the same path."
Forbes read the post as OpenAI calling time on the industry's experimental phase, and put the pricing logic plainly: "Think of it like a consulting firm that takes a percentage of the savings it generates, rather than billing hourly. If an AI agent helps a pharmaceutical company discover a new drug faster, OpenAI wants a cut of that value." In July the company's scorecard post said "the lowest price per token does not always produce the lowest cost per outcome" and proposed "Useful Intelligence per Dollar" as the number that matters. In the same post it advertises GPT-5.6 Sol using 54% fewer output tokens than a rival. Token efficiency as marketing, and outcome pricing as the exit from tokens.
If the provider is paid for completing the task rather than for every token generated, spending additional internal compute becomes worthwhile whenever the increase in success probability is worth more than the additional inference cost.
The optimization target shifts from:
How cheaply can we generate a token?
to:
How much compute should we spend to produce the most profitable successful outcome?
That creates a strange consequence for model evaluation. And sets us up for a future where two possible outcomes might happen: models increase in cost as the strategy to "get you hooked" (lower initial cost) flattens out, and the provider changes strategy to profit maximization. Or we get a completely different incentives model built around outcomes.
A model that appears to consume fewer tokens may genuinely be more efficient. But the number exposed by the API may also become an increasingly incomplete proxy for the amount of computation actually spent producing the result.
The leaderboard makes the point on its own. Its cost column moved four times in two months without a single new run: OpenAI cut prices on 30 July and again on 21 August, Google on 13 August, and a DeepSeek cut had been counted twice. Sol dropped from $8.39 to $6.46 per task on the same 60k tokens. Gemini 3.8 Flash's $2.36 doubles on 1 January.
As the economics of intelligence progresses we will see different strategies being tested and iterated. Currently, I look at total output tokens with a grain of salt, and the cost of the model as an impermanent fact.
Sources and measurement
The figures use the DeepSWE v1.1 leaderboard as generated by Datacurve on 3 September 2026 and retrieved on 7 September. Two records of it are saved: the public artifact and the 70 configuration rows embedded in the page. They share one generation time and disagree on cost for 19 configurations, because the page reprices models whose list price changed after their runs and the artifact does not. The chart dataset plots the page's cost, keeps the artifact's figure beside it wherever the two differ, and records both file hashes.
Every point is one model at one reasoning effort, with its mean pass@1 over four whole-benchmark runs, its mean cost per task, its mean output tokens per task and its mean agent steps. Nothing is excluded. The three highlighted configurations are picked by one rule: every configuration whose pass rate rounds to the same whole point as the leader's, which is the tie the leaderboard itself prints.
Only the GPT-6 Astra rows break the bill into cache reads, cache writes, reasoning tokens and compute units, and only they state a price basis. The dataset script checks that the stated rates applied to the stated token counts reproduce the reported cost to the cent, which they do only when reasoning tokens are counted inside output tokens. So for Astra the output-token column on the leaderboard includes billed reasoning. The basis DeepSWE states for Astra is above OpenAI's published rates and carries a compute-unit line the pricing page does not list. The dashed mark on each bar is the same tokens at the published rates without that line. Which basis Astra is actually billed at is not settled by anything I could find, so the essay quotes DeepSWE's number and shows both.
All three figures are also available as images: tokens, cost and bill, with an SVG original and a chart-only variant beside each, and a manifest recording the dataset hash.
