What matters in AI this week

By Kenrick Cai, Technology Correspondent

CEO of OpenAI Sam Altman speaks during the 2026 Infrastructure Summit of government officials, corporate executives, and labor leaders, in Washington, D.C., U.S., March 11, 2026. REUTERS/Kylie Cooper/File Photo

If you’re trying to understand how AI companies actually make money, tokens are starting to matter more than how many users are engaging with any given AI model.

At a basic level, tokens are just units of data. They measure the amount of work a model does when it processes a request. But in business terms, they’ve become a proxy for usage and, crucially, revenue. Anthropic’s staggering growth, despite its Claude chatbot trailing ChatGPT in consumer usage, can be attributed to developers using tokens at rates that are dozens of times those of the average consumer.

That has propelled its coding platform Claude Code to billions of dollars in revenue. Altman’s Codex announcement yesterday is OpenAI’s latest attempt to play catch up. Simply put, the AI labs are benefiting from how much their power users engage with the technology, rather than just sheer user numbers. But that surge is also drawing attention to the costs of such profligate consumption.

During the livestream on Tuesday, Altman suggested that developers talking about blowing their entire annual token budget in a quarter was approaching meme territory. In March, Nvidia CEO Jensen Huang proposed at a company conference that engineers should be getting paid in tokens as part of their compensation.

“If that $500,000 engineer did not consume at least $250,000 worth of tokens, I am going to be deeply alarmed,” Huang added during a podcast appearance around the same time.

Altman said the heaviest user among OpenAI’s employee ranks has increased token usage from 100,000 per month to 100 billion over the last 6.5 years. That is a 1 million-fold increase. 

But the mood is starting to shift. There’s increasing backlash around heavy token usage, and mounting unease among companies actually footing the bill despite the staggering revenue growth.

The panic comes in part because business leaders are having trouble tracking AI use, Russell Kaplan, president of coding startup Cognition, told Reuters.

Some companies scrambling to measure the value of their AI investments have relied too heavily on token usage, Kaplan said. But the metric does not do as good a job approximating revenue gains for users as it does for the model providers.

In other words, just because you’re using more AI doesn’t mean you’re getting more out of it.

“You should not be maximizing token usage for its own sake,” Kaplan said.

That concern is starting to show up in how companies talk about AI spending more broadly. What was barely a line item not long ago is now a regular talking point in boardrooms. Altman acknowledged as much, noting that OpenAI is shifting more attention toward costs.

Google, which is also making an enterprise push, used that exact argument to pitch businesses on switching to its Gemini model from more expensive rival options.

“We've heard that many companies are already blowing through their annual token budget; it’s only May,” Pichai said during the keynote for Google’s annual I/O developer conference. The most AI-forward companies stood to save upwards of $1 billion per year, Pichai said, if they shifted the majority of their workloads Google’s way.

All of which sets up an interesting backdrop for the coming IPO wave.

Investors won’t just be betting on who has the most users or the flashiest models. They’ll be asking a more basic question: Who actually makes money, and at what cost?

Increasingly, the answer to that may come down to tokens.

This newsletter was edited by Ken Li, Rosalba O'Brien and Edmund Klamann. Additional reporting by Deepa Seetharaman.