|
Big Technology is possible thanks to support from our readers. Sign up today to help us do this work and gain access to perks like members-only articles and our private Discord server:
The Token Reckoning is Here and It’s Not What You Think
Token waste in service of productive use, not waste for the sake of waste, could be the biggest problem in AI right now.
|
| |||||||||||||||
 |
In June 2025, OpenAI CEO Sam Altman wrote that “Intelligence too cheap to meter is well within grasp,” predicting that as “datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity.”
Exactly a year later, AI costs are starting to look less like cheap intelligence and more like an AC bill in the middle of a heat wave. Stories of companies blowing through their annual token budgets in months abound, and at least some companies are rethinking (or cancelling) major AI initiatives.
But while recent headlines about ‘tokenmaxxing’ and runaway budgets have caused concerns that AI’s momentum has been inflated, the panic might be misplaced.
Token waste in service of productive use, not waste for the sake of waste, could be the biggest problem in AI right now. For every $1 in AI spend, 82 cents never make it to production, according to the startup EntelligenceAI, which estimates only 18 cents turn into shipped product, with the rest going to bug fixes, rewriting, or reworking code, and review processes.
If even directionally accurate, EntelligenceAI’s numbers are a major warning sign for the AI boom. Tokenmaxxing can be fixed by removing leaderboards or monitoring budgets closely. Unproductive uses of the technology can only be solved by finding productive uses.
This is why Uber COO Andrew Macdonald saying last week that it was tough to justify all the AI spending, because it wasn’t leading directly to productivity, went viral. If Uber’s experience is representative, that would signal an era of slowing growth ahead.
To be sure, negative AI stories seem to travel further than positive ones, and many companies are finding productive uses of the technology. So some caution is warranted. But ultimately, the industry will have to sort out the ROI question or risk a significant comedown.
The Right Way To Build AI Agents — With NVIDIA’s Adel El Hallak and ServiceNow’s Joe Davis (Sponsor)
 |
Joe Davis is the EVP of AI Engineering, and Delivery at ServiceNow. AdelEl Hallak is the VP of Product Management, Agentic AI at NVIDIA. Both join for an episode on the inner workings of building AI agents. Tune in to hear how both are working together to make autonomous AI agents safer and governable, how ServiceNow's L1-AI-IT specialist has automated 90% of internal support tickets, and what the next few years hold for agents. Hit play for a behind-the-scenes look at what it actually takes to deploy AI agents in the enterprise.
The Intelligence Report
Microsoft Build is this week on June 2nd and 3rd and will likely include a range of new AI-related news and other updates for hardware and software. Expected news includes new homegrown AI models and updates for making Copilot a superapp that could help it compete with ChatGPT, Claude, and Gemini.
In a speech today at a WAN-IFRA conference, New York Times publisher A.G. Sulzberger escalated the media industry’s fight with AI companies, warning that OpenAI, Google, Meta and Perplexity are undermining journalism by repackaging publishers’ work, siphoning off traffic and revenue, and making it harder for news organizations to fund original reporting.