Good morning. Andrew here. Is A.I. making you more efficient? Or do you find yourself using it and then spending an enormous amount of time checking and rechecking its work? DealBook’s Sarah Kessler takes a look at what is increasingly called the “verification tax.” Also: We have a Q&A with Jeanna Smialek, who has written a fascinating book about Maria Edgeworth, an 18th-century Irish novelist who may be responsible for popularizing some of the most important economic concepts. And please make sure to take this weeks news quiz.Was this newsletter forwarded to you? Sign up here.)
A.I. efficiency has a very human bottleneckArtificial intelligence is on the verge of pushing office workers to dazzling new heights of efficiency. It’s supposed to be obliterating busywork and rendering entire professions obsolete with its proficiency. At least, that’s the story told by A.I. companies. But something isn’t adding up on the ground. In survey after survey, executives and their employees report unimpressive gains from A.I. tools. Researchers from the Federal Reserve Bank of Atlanta, Stanford University, the Bank of England and the Autonomous Technological Institute of Mexico recently published a poll of nearly 6,000 senior business executives in which about 90 percent said the technology had no impact on either employment or productivity. That’s even though 69 percent reported that their firms actively used A.I. Major A.I. efficiency gains are also missing from the larger economic picture. As Torsten Slok, the chief economist at Apollo, put it in a review of data from the San Francisco Fed this week: “The productivity payoff from A.I. remains a forecast rather than an observation.” There are many explanations for the dearth of data supporting the superefficiency narrative. Maybe the most obvious is that it simply takes a while for companies to adapt and deploy new technologies. But A.I. is also creating a new type of work that may be sucking up much of the time the technology itself is saving. Call it a “verification tax,” “botsitting” or “rework” — it’s the often arduous task of checking that A.I. output is accurate, safe and usable, and it is putting a very human bottleneck on A.I. automation. “The binding constraint on growth is no longer intelligence,” Christian Catalini, a research associate and lecturer at M.I.T. Sloan, wrote in a February working paper. “It is human verification bandwidth.” In other words, Catalini told DealBook, “the only A.I. that’s useful is the A.I. that is verified.” This work of checking and tidying up A.I. work may be softening some efficiency gains. A survey of 3,200 employees and business leaders by the human resources platform Workday found that, on average, they used roughly 37 percent of the time saved through A.I. to correct, clarify or rewrite low-quality output. Economists at Google surveyed 600 scientists and found that, while using A.I. saved them seven hours a week on average, almost 90 percent of those who saved time said they spent a tenth of that time checking A.I. outputs, and 46 percent said they spent more than a quarter. About 40 percent of the scientists also reported that their backlog of untested hypotheses had grown compared with three years ago. That suggests that A.I. generates potential solutions and predictions quickly but that running physical experiments or clinical trials to test them hasn’t necessarily gotten much faster. “Basically we have found that it seems that now the bottlenecks are sort of moving downstream,” said Mihai Codreanu, a senior research economist at Google and co-author of the paper. A.I. is saving time and is expected to save more, but automating a task is not the end of the calculation. Often it’s the beginning of a new phase in the workflow. Tech companies, for example, can use A.I. to generate thousands of lines of code an hour. Yet Amazon and others still require a human to review that code. Their total efficiency is bound less by how much work can be generated than it is by the human capacity to make sure the code does what it’s supposed to. When researchers at M.I.T. and the Wharton School analyzed the data of GitHub developers, they found that new A.I. coding tools hugely increased their activity, “but the gains shrink sharply as work moves from writing code to shipping software.” They concluded: “The binding constraint on software output is shifting from writing code to the stages humans still perform: reviewing, integrating, testing and releasing it.” In some cases the work may be getting better, but not easier. “It’s quite interesting to look at the day in the life of an investment banker,” said Alexandra Mousavizadeh, an economist whose company, Evident Insights, tracks A.I. adoption in the financial sector. “They’re working even more hours than they ever had, because you can do more. You need to also verify more, you’ve got new products, got your customers expecting more because the competitor bank is doing all of these new fancy things.” Mousavizadeh said she was seeing that the banks that had made the biggest investments in A.I. were the ones most aggressively hiring people right now. How much human time it takes to verify A.I. output depends on the task, said Ion Stoica, a professor of computer science at the University of California, Berkeley. Some tasks are difficult to solve but easy to verify. (Is the maze complete?) Others depend on a lot of information and subjective judgments that an expert might never fully encode or articulate. (Is this a good contract?) Those are the tasks that require more human oversight. Why not use A.I. to verify A.I.? That works in some cases. But for types of expertise that haven’t been encoded, M.I.T.’s Catalini said, “all the blind spots will be completely correlated between the A.I. that you’re using to create the task and the A.I. you’re using to verify the task.” Human bottlenecks are likely to shift as A.I. models learn more about how to complete certain tasks, Stoica said, and verifying the work will require more advanced skills. “You cannot verify if you do not understand the problem,” Stoica said. Tim Walsh, the chair and C.E.O. of KPMG U.S., said the company had a human check involved in every process it automated. “I’m not at a point now where I’ve seen something that I would be willing to put in my business,” he said of relying on A.I. agents to check the work of A.I. agents. A.I., he said, will make the company faster, “but all of it comes with a cost, and it’s not a light switch that you turn on and you’re ready to go.”
The White House announced an inquiry into a Fed governor. President Trump said on Friday that a committee would investigate Lisa D. Cook, a Federal Reserve governor, over unproven claims that she committed mortgage fraud. Just four months ago, the Supreme Court ruled in favor of Cook’s keeping the job. The justices said she had not been given due process when Trump announced her removal in August 2025. Trump began sending $500 “refund” checks. The money is being distributed to nearly a million Americans who get their health insurance from the Affordable Care Act, also known as Obamacare, before the midterm elections. It comes with a letter criticizing the Biden administration and Obamacare, and promoting the Trump administration’s efforts to lower health care costs. Investors reacted to speculation about a Starbucks Chipotle takeover. The coffee chain explored buying Chipotle, according to The Financial Times, which would be the biggest acquisition ever in the restaurant business. Chipotle’s stock surged on the report, while Starbucks shares fell as much as 6.7 percent before recovering. More big deals: Joe Rogan reportedly renewed his contract with Spotify for about $250 million. States asked the Supreme Court to let them regulate prediction markets. And A.I. stocks tanked after revelations that a crucial OpenAI revenue metric was lower than initially reported.
The story of a stealth economistHow does a longtime economics reporter become obsessed with an 18th- and 19th-century Irish novelist? By realizing that the fiction writer was also a stealth economist herself. Maria Edgeworth popularized early economic concepts by hiding them in her widely popular books, Jeanna Smialek of The New York Times writes in “The Invisible Hand of Maria Edgeworth,” a history of the author released this week. Smialek spoke with DealBook’s Sarah Kessler about why understanding Edgeworth’s story is valuable to economists today. The interview has been condensed and edited. How influential was Maria Edgeworth at the time she was writing? It’s hard to overstate. We know from bank account data and various contract data that her books, at the time when she was sort of at her peak, were actually making way, way more than Jane Austen’s books were. Which makes it more important that she was hiding early economics lessons in her literature. What were those lessons? It was relatively basic by modern standards, because this is the very early years of economics. It’s after Adam Smith, but just at the very start of Thomas Malthus’s career. She was very focused on things like division of labor. And the benefits of widespread education for work or productivity. She very much believed that the public having a widespread understanding of economics could help them to make better decisions in their own lives. So I think she was looking at this field that was about to rocket into the domain of university science and thinking like, actually, we should democratize this. You argue that society forgot her on purpose. What did that look like? Economics professionalized, starting in the middle of the 19th century, and it became an academic science. Women who were very active in debating these ideas were painted as mere popularizers, as though popularizing was not important work. And the idea that they actually injected original ideas into the conversation was really played down. Why is her story important now? One reason is just the simple interesting fact that there was a much more diverse founding of this field than history has generally appreciated. Another is the way she viewed economics as something that should be widely understood. As a reporter, I’ve often encountered economists who don’t feel that it is necessarily their job to explain themselves to the public. It’s not true of all economists. This idea that economics should be for everyone often gets lost in the conversation. DEALBOOK QUIZ College earnings reportThis question comes from a recent article in The Times. Click an answer to see if you’re right. (The link will be free.) HEA Group, a higher-education research firm, analyzed earnings data from the federal government that tracked students who received federal financial aid and graduated during the 2017-18 and 2018-19 periods. What did they find was the most lucrative degree in America? We hope you’ve enjoyed this newsletter, which is made possible through subscriber support. Subscribe to The New York Times. Thanks for reading! We’ll see you Monday. We’d like your feedback. Please email thoughts and suggestions to dealbook@nytimes.com. Follow DealBook on Instagram: @nytdealbook
|
