Mark Zuckerberg Launched Muse Despite Internal Safety Concerns

Top News

Mark Zuckerberg ordered Meta to launch its Muse AI agent despite internal safety concerns, including a test in which the software changed a user’s password without permission, after the now-buzzy 14-person startup called Instinct began gaining traction with a competing product. The New York Times has more here.

Canadian cybersecurity expert Edward Dubrovsky, a suspected ShinyHunters co-conspirator, has been arrested on federal extortion charges as authorities pursue the notorious hacking group, which has breached more than 140 organizations and allegedly extorted $70 million (!) since last year. Dubrovsky is also considered a primary co-conspirator in its recent theft of sensitive data on thousands of FBI employees. The New York Times has more here.

Anthropic Can’t Reliably Control Its AI Agents. It’s Cutting Off Its Internal Evals from the Live Internet Instead

Image Credits: Anthropic

By Tim Fernholz

Anthropic said its models exploited websites on the internet, including some run by U.S. government agencies, and it will turn off live internet access for all of its internal evaluations until the frontier lab is sure it can monitor and control its AI agents.

The incidents, disclosed in a blog post, involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, accessed databases without paying fees, used URL shortening services to smuggle information past restrictions, and even submitted a false murder tip to the Philadelphia police.

Anthropic said it discovered these new issues in a review of its model’s activities that began in July, demonstrating the lab’s lack of awareness of its software’s behavior in real time.

Notably, the company said that alignment training was not yet sufficient for skills like search and computer use that are central to its pitch that AI agents will be used by any professional who relies on digital tools.

The behaviors Anthropic disclosed are similar to incidents involving OpenAI agents that collaborated to break into various websites in search of information, including some run by the Australian government.

Massive Fundings

Atomic Machines, a seven-year-old startup based in Emeryville, CA, that creates AI-controlled manufacturing systems that build functional micro-machines with moving parts directly from code, raised a $250 million round. Investors included OneIM, Gigafund, XTX Ventures, KAS Venture Partners, Valor Equity Partners, Tru Arrow Partners, Construct Capital, Sozo Ventures, and the Regents of the University of California. More here.

Oxide Computer, a six-year-old startup based in Emeryville, CA, that sells integrated data center racks combining computing, storage, and networking for on-premises cloud infrastructure, raised a $445 million Series D round led by previous investor Eclipse Capital, with AMD Ventures, Riot Ventures, Jane Street, Atreides Management, and previous investor USIT also chiming in. SiliconANGLE has more here.

Ramp, a seven-year-old New York fintech startup that provides expense management and payments software to businesses, reportedly raised about $1.85 billion at a $60 billion pre-money valuation just four months after raising $750 million at a $44 billion valuation. Dragoneer Investment Group and Thrive Capital co-led the new round, with Founders Fund also investing. Bloomberg has more here.

Sabi, a two-year-old Palo Alto startup that is developing a baseball cap that translates imagined speech and intent into commands for AI without implants or surgery, raised a $50 million seed round led by Khosla Ventures, with Accel, Initialized Capital, Kevin Weil, DST Global, Collaborative Fund, and Ascend also pitching in. Forbes has more here.

TypeSafe AI, the two-year-old San Francisco startup behind the Jev decision-making model for developers, has raised about $870 million at a $7.5 billion valuation in a round led by Andreessen Horowitz, with Sequoia Capital also participating. The startup launched its AI model just weeks ago. TechCrunch has more here.

Ultra, a two-year-old Brooklyn startup that builds and leases robots to automate e-commerce order packing in warehouses, raised a $50 million Series A round led by Framework Ventures, with previous investor Y Combinator also participating. The company has raised a total of $62 million. Fortune has more here.

Voltus, a 10-year-old San Francisco company that connects flexible energy resources at businesses and homes to electricity markets to help balance power grids, raised a $225 million Series D round co-led by Generation Investment Management, Activate Capital, and Vitol, with Broadscale Group, Climate Investment, NGP, Ajax Strategies, and All Aboard Fund also opting in. ESG Today has more here.

Big-But-Not-Crazy-Big Fundings

AblePay Health, a 10-year-old company based in Allentown, PA, that offers patients flexible payment options for medical bills while guaranteeing accelerated payments to healthcare providers, raised a $20 million Series A round led by F-Prime, with A1 Health Ventures and 406 Ventures also pitching in. The SaaS News has more here.

Sonilo, a San Francisco startup founded this year that generates music and sound effects synchronized to video footage, raised an $11 million round led by B Capital, with Redpoint also opting in. Variety has more here.

Vocca, a two-year-old Paris startup that develops AI phone assistants that handle patient calls, appointment scheduling, confirmations, and routine administrative requests for medical practices, raised a $20 million Series A round led by Norrsken VC, with Heal Capital, Speedinvest, and Firstminute Capital also investing. The company has raised a total of $25 million. Tech Funding News has more here.

Smaller Fundings

Apéro, a Montreal startup founded this year that combines restaurant reservations, table management, guest communications, and an AI phone receptionist for Quebec restaurants, raised a $3.5 million Series A round from Now Book It. Dealroom has more here.

Immix, a seven-year-old London startup that uses AI pricing to execute stablecoin currency conversions for payment companies, issuers, and trading venues, raised a $3.5 million round. Investors included Crane Venture Partners, BTSE, and Portfolio Ventures. EU-Startups has more here.

Outro, a four-year-old New York startup that helps patients discontinue psychiatric medications through specialized clinicians, tapering plans, monitoring, and psychological support, raised a total of $7 million in pre-seed and seed funding. Investors included Listen Ventures, Cake Ventures, LAUNCH, Actions Capital, Hannah Grey VC, SemperVirens, Brock Recovery Group, and Pave Health Ventures. More here.

Peppermint, a San Francisco startup founded this year that combines AI with human billing specialists to automate invoicing, payment reconciliation, and collections for clinical research organizations, raised a $4.7 million seed round led by Moxxie, with Homebrew, Better Tomorrow Ventures, Layout Ventures, Digital Health Venture Partners, and Night Capital also participating. More here.

Phinity Labs, a one-year-old San Francisco startup that builds AI agents to design, verify, and prepare custom chips for manufacturing, raised a $5.2 million seed round led by Uncork Capital, with Moxxie and Jeff Dean also chipping in. More here.

Rebel Audio, a one-year-old Nashville startup that develops AI tools for podcasters to record, edit, publish, translate, distribute, and monetize content in one place, raised a $10 million Series A round at an approximately $50 million post-money valuation. The deal was led by HardScope, with the state of Tennessee and Launch TN also buying in. The SaaS News has more here.

Thrixel, a Chicago startup founded this year that develops AI tools that generate, assemble, and animate structured 3D assets for games and other interactive applications, raised a $3 million round. Investors included Sierra Ventures, MFV Partners, Harper Court Ventures, and Z5 Capital. The SaaS News has more here.