Technology

1. Semafor Tech’s new show
2. Demands for AI safety legislation
3. China’s AI agents
4. Rogue agents go global
5. Unreal influencers

The West’s open-source AI race is on, and OpenAI says its models have solved hundreds of math problems.

The race in the West to become the top open-source AI model shifted into higher gear this week, as both Nvidia-backed Reflection AI and French lab Mistral unveiled new models they hope will break China’s open-source monopoly.

The challenge looming over Beam and Le Chonk’s debuts (immediately my new favorite model name) is that they lag the top Chinese models and the leading closed systems from OpenAI and Anthropic on prominent benchmarks.

But Reflection and Mistral boasted in their announcements that they do beat other open Western models, such as those from Thinking Machines and Nvidia, suggesting the companies care just as much about the race against each other as the one with Beijing — despite US and European companies’ shifts to using more Chinese models over the past year.

Reflection CEO Misha Laskin told me his goal isn’t necessarily to dismantle the foothold Chinese models have achieved among developers and large companies. Instead, he sees a market in entities that can’t or won’t use Chinese models — think heavily regulated industries and governments — and might otherwise have no choice but to pay for closed American AI.

The timing of that pitch is opportune: Many businesses are heading into year-end budget reviews with eye-popping AI spend figures to account for. Governments are also bracing for sticker shock as $1-a-year pilot deals expire. New data from Ramp suggests companies are switching which AI providers they use more than ever.

The question that will determine the fate of these and other Western open-source players is whether businesses and agencies will tolerate less advanced models for some tasks in exchange for affordability and control.

Introducing ‘Future Proof’

Technology is at an inflection point. From AI to the forces reshaping business, science, and daily life, the pace of change is accelerating. Our new show, Future Proof from Semafor Technology, offers a direct look at the breakthroughs, risks, and questions defining what comes next. Hosted by Reed, each episode features conversations with the builders, policymakers, and industry leaders shaping how technology is developed, adopted, and governed.

In the debut episode of Future Proof, Cognition CISO and former Facebook security chief Alex Stamos goes off on the “nihilism” ripping through Silicon Valley. He joins Reed to discuss the very real risks of AI, and the wholly imagined ones shaped by “almost religious beliefs” held by people at AI companies. “These things are not conscious, they do not have souls,” he says.

Stamos says he plans to “work the problem” to help shorten the inevitable “dark age” of cybersecurity. “What you should not do is just quit your job and freak everybody out,” he says.

Plus, why no one would work for Mark Zuckerberg if he resembled Aaron Sorkin’s celluloid version in the upcoming film The Social Reckoning.

Progressive coalition lays out AI demands

A coalition of nearly 40 labor, progressive, faith, and AI safety groups is laying out demands for AI legislation in Congress, warning lawmakers that any bill that crosses their red lines will face “active, mobilized opposition,” according to a letter shared first with Semafor, Ashley Gold scoops. The letter is spearheaded by Guardrails Action, the nonprofit affiliated with the Guardrails Alliance super PAC launched in June as a counterweight to pro-AI election groups.

The group is calling for “independent, transparent government oversight” and “a mandate that addresses human impacts from safety to economic disruption.” They also want to ensure that “AI models are not making final decisions” on issues like healthcare and weapons. “We want members of Congress and 2028 candidates to know that it’s not enough to come to the table with half measures,” said Maya Handa, a senior adviser at Guardrails and former adviser to New York City Mayor Zohran Mamdani.

How the AI agent business is different in China

US and Chinese tech companies are taking differing approaches to bring AI agents to the masses, the industry’s latest fixation. OpenAI’s recently released dots and Meta’s Muse are built around the assistant first and then sent out to use other companies’ websites and services.

Chinese counterparts, including Tencent’s Xiaowei — which is built into super-app WeChat — can, for example, place orders from businesses that are already on the app. It’s an obvious use for closed systems like WeChat and Alipay, which Chinese consumers already use for much of daily life.

Xiaowei’s strategy may not translate into revenue, the Hello China Tech newsletter wrote: Tencent earns no new fees from purchases, and early tests show it making mistakes. It begs the question whether the super-app strategy limits how useful Chinese agents can become since Xiaowei can only assist with what’s available inside WeChat, and other Chinese platforms have blocked agents from outside.

Muse and dots may not be a seamless experience either, given Muse has already been blocked by Amazon and both require users to turn over access to private systems, but their popularity already suggests markets are watching closely.

— J.D. Capelouto

Rogue agent whack-a-mole

Governments around the world are hardening their AI stances as more agentic hacks breach critical systems. South Korea’s president noted “considerable public concern and anxiety” following revelations that hackers used an AI system to execute recent cyberattacks against banks in the country.

And officials from OpenAI and Anthropic faced questions from lawmakers in Australia on Tuesday, after the ChatGPT maker drew criticism for not alerting Canberra fast enough after one of its models gained unauthorized access to a government health data portal during internal training and evaluation.

Tech companies are playing whack-a-mole around the world as their models get up to more hijinks and are used for malicious purposes. (It’s unclear what specific AI was used in the South Korea hack, though hackers reportedly used a Chinese-made open-source agent that can run on models from Anthropic, OpenAI, or DeepSeek.)

Insurers are also taking note, and preparing for multimillion-dollar lawsuits over rogue AI agents — and weighing whether executives could be held liable, the Financial Times reported.

— J.D. Capelouto

AI influencers get weirder

There’s a new kind of AI influencer flooding social media feeds — and they’re not even pretending to look like real people. Higgsfield — which recently announced it hit a $1 billion run rate, and which boasts 30 million users — has updated its AI influencer tool, making it easier for users to insert a character of their own creation into existing videos. The characters are peculiar: They look realistic, but not quite of this world. Many have geometric haircuts, elongated necks, or other extreme bodily proportions that make clear they’re artificial. That’s by design. “A flawless face is the fastest way for a character to read as generic AI,” a Higgsfield spokesperson told Semafor.

This odd aesthetic plays to AI’s strengths: Creating uncanny characters is a better use of the technology than trying to pass as human. It’s a more transparent way to create content that doesn’t deceive the audience the way deepfakes can. Some of these AI-generated influencers have racked up millions of views and thousands of followers across platforms (some users are already using them to promote memecoins).

— Jake Angelo

MIT Technology Review’s 10 Climate Tech Companies to Watch list features the startups and established businesses our newsroom believes have the best potential to address the threats of global warming. Subscribe to find out who made this year’s list, plus get bonus AI content.

In Friday’s edition, we reported that OpenAI’s Astra model had cracked a 217-year-old Napoleonic cipher. The company has now released a list of more than 300 mathematical proofs it says its AI models have solved, posting more than 700 pages of work on GitHub, which mathematicians are parsing through to confirm. AI’s mathematical breakthroughs have sparked debate across the math community about what to do if AI surpasses human capabilities in the field, and many are still grappling with the technology’s most recent discoveries. “If a human did this, it would be an instant Fields Medal, no questions asked,” a math professor told The Wall Street Journal.