DeepSeek reduced prices on its V4-Pro model by 75%, aiming to benefit enterprise AI vendors and developers. The move didn’t deliver the expected financial relief for many.
The challenge isn’t just cost—it’s usage. While inference prices fall, agent systems consume tokens at a rate that exceeds those savings. A single user request can trigger dozens of internal operations, turning a simple chatbot interaction into a financial burden.
The 100x problem
Software economics once followed a predictable pattern: infrastructure became cheaper while applications grew more capable. AI was expected to follow suit. That expectation is now unraveling.
A traditional chatbot converts one user question into one model call. An agent, however, chains planning, retrieval, tool use, verification, and follow-up decisions. The user sees a single answer, but the vendor pays for the entire process. This creates a 100x problem—where one visible request can cost 100 times more to serve as an agentic workflow than as a simple chatbot response.
In extended workflows, the multiplier increases. Declining model prices offer some relief, but they don’t address an architecture that turns one prompt into dozens of billable operations.
Related: AI coding tools spawn new software supply risk
Token amplification in action
A seemingly straightforward agent query like, “What did our top customer ask about last week?” typically involves seven priced operations:
- User prompt (~50 tokens)
- System prompt and tool definitions (~3,000 tokens, repeated on every call)
- Retrieval (~5,000 tokens of context)
- Model call #1 — tool selection (8,000 in / 200 out)
- Tool execution (~4,000 tokens returned)
- Model call #2 — summarization (12,000 in / 400 out)
- Model call #3 — follow-up decision (12,400 in / 100 out)
One sentence from the user results in roughly 35,000 input tokens billed. At current frontier model rates, that translates to between $0.10 and $0.40 per query. For a company handling a million queries monthly—the baseline for enterprise B2B features—costs reach six figures.
The scenario isn’t theoretical. OpenAI’s program offering Y Combinator startups $2 million in API credits, once enough to fund an entire seed round, now reflects the actual expense of running an AI-native company through its first year.
Established enterprises face even larger numbers. Several vendors are now privately reporting negative gross margins on heavy users.
The situation creates an unusual pattern: customers deriving the most value from AI often generate the highest costs. In extreme cases, vendors discover their most engaged users are the least profitable.
Related: Enterprise AI Faces Growing Debt Risks
The core issue isn’t just AI’s expense—it’s that the dominant business model for enterprise AI can’t sustain agentic workloads. Seat-based SaaS pricing assumes a reasonably bounded cost-per-user. Token amplification shatters that assumption.
This isn’t the first time infrastructure costs have disrupted business models. Early cloud computing forced companies to rethink pricing and scaling. The difference now is the speed at which costs escalate—before pricing models can adapt.
How companies are responding
Technical solutions exist but aren’t always simple to implement:
- Cost-aware routing: A small classifier model determines which tier of model handles each query. Well-tuned routers can reduce inference bills by 60% without sacrificing quality.
- Context discipline: Truncating tool outputs, pruning reasoning traces, and capping tool depth prevent agents from excessive operations.
- Speculative decoding: For self-hosted deployments, this method can triple effective throughput on the same GPUs.
Firms implementing these measures begin to resemble financial trading systems. Every routing decision carries a price, every path has its own P&L, and every tenant operates on a metered budget.
IBM’s findings show organizations using orchestration-led governance achieve six times greater productivity impact than those relying solely on compliance. The advantage isn’t just efficiency—it’s survival.
Related: AI model achieves efficient memory usage
Four key actions separate companies that will maintain margins in 24 months from those that won’t:
- Treat inference cost as a primary metric. Monitor it per-feature, per-tenant, and per-query class, as cloud costs were tracked in the mid-2010s.
- Budget like a media buyer. Set cost-per-thousand-queries limits per feature. Enforce caps and alert on overruns. Engineering teams won’t do this alone.
- View the router as essential infrastructure. It’s no longer an optimization—it’s the new load balancer.
- Audit prompts quarterly. A 4,000-token system prompt that expanded over six months can become a six-figure expense. Most teams haven’t reviewed their production prompts end to end.
Early volume commitments also help. Frontier-model vendors now offer reserved-instance-style prepaid commits at significant discounts. List price remains the worst rate any enterprise will pay.
The structural shift isn’t about AI being expensive. Frontier inference costs drop roughly threefold each year, and the trend continues. The problem is that amplification outpaces those price cuts. A 75% reduction in per-token costs doesn’t benefit a company whose agents process 700 times more tokens per user query than its pricing model anticipated.
For the first time since the cloud era began, architecture choices directly impact finances in real time. A prompt redesign affects margins. A poorly controlled agent loop becomes an outage with a credit card attached.
The companies that endure won’t be those using the cheapest model. They’ll be the ones whose agents are efficient—and understand their own operational costs.
Leave a Reply