|
 |
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here.
Inside OpenAI’s agentic software factory
A deepdive into how Codex has “taken over” OpenAI, how the frontier lab builds its agentic software factory, and the engineering challenges of one billion users. Details from inside OpenAI
|
| |||||||||||||||
It’s rare to work with an unlimited token budget, but at OpenAI, that’s what all engineers, researchers, finance colleagues, and marketing folks do. Recently, I visited one of the world’s leading frontier labs to find out how OpenAI operates today – and for a glimpse at where software engineering might be headed as a profession.
Plenty has changed since I visited OpenAI’s headquarters last year. Within a year, Codex has gone from a “nice-to-have” tool to being the backbone of pretty much everything at the company.
To learn more, I talked with seven engineering leaders and engineers there: Venkat Venkataramani (VP of Engineering, Applied Infra), Sulman Choudhry (Head of Engineering, ChatGPT), Andrew Ambrosino (Lead, Desktop), Joe Gershenson (Lead, Core Agent team), Akshay Nathan (Engineering Lead, Productivity), Ahmed Ibrahim (Engineer, Codex) and Steve Coffey (Engineer, Responses API). Thanks to all for taking part!
Today, we cover:
Codex takes over at OpenAI. In a matter of months, nearly all OpenAI’s non-engineers moved over to Codex and ChatGPT Work without a mandate from above for it.
Death of the IDE & pull requests. IDE usage has been down since January when Codex usage started to surge. PRs and code reviews need to be rethought.
OpenAI’s agentic software factory. OpenAI has built a “software factory” with several automated, agentic feedback loops: for example, Perf Factory monitors production and kicks off Codex agents to automatically fix performance issues.
How engineering tooling & practices are changing. Hand-built internal tools are slowly being replaced by Codex, which is increasingly preferred for debugging over specialized tools. Harness efficiency is critical in software factories.
Engineering for a billion users: how OpenAI scales up its infra. They buy first and take it in-house later. Also, geographic infra distribution, capacity planning tactics and challenges.
Making OpenAI’s API more reliable and performant. CPUs are becoming a bottleneck, doing slower deployments on purpose, and solving load challenges.
How the software engineering job is changing. Engineering specializations are disappearing, judgment and agency are more important, and it only takes one or two engineers for previously “impossible” rewrites and migrations to succeed.
Before we start, a scheduling update: I’m in New York for the week, attending the LDX3 conference and visiting a few startups and tech companies in the city, so there will be no edition of The Pulse on Thursday. Normal service resumes next week!
The bottom of this article could be cut off in some email clients. Read the full article uninterrupted, online.
1. Codex takes over at OpenAI
The takeaway from my visit to the company’s headquarters which really sticks out is that Codex – and lately Codex and ChatGPT Work – have taken over everything there, starting in around January. Desktop lead, Andrew Ambrosino, told me:
“The big theme of the past months has been that everything is now a coding agent. Whether the visible code is your output or not, agents write your artifacts.
Think of it like this: your entire life is via software. You have these powerful tools (agents) in your computer, and the ability to loop and reason and write code is the ability to do everything.”
The token usage chart below shows this sudden adoption surge:
 |
In a four-month period, non-engineering orgs like finance, recruitment, and legal went from ~0% usage of Codex to 90% usage. Now, almost all OpenAI employees use Codex and ChatGPT Work weekly. So, what happened?
OpenAI released the Codex app for Mac in February and for Windows in March, and ChatGPT Work (powered by the Codex harness) in July. Following that, non-engineers there moved all their workflows over to Codex and then to Work. Caveat: OpenAI’s internal version of Codex is a lot more advanced than its external counterpart because it’s plugged into pretty much every OpenAI system – similar to how Ramp’s Inspect AI agent has been wired up.
The fascinating part of this is that OpenAI got close to 40% adoption across non-engineering teams at a time when the Codex app was hostile to non-engineering users (hard to use). Between February and April, the Codex app still showed the code on-screen, but even so, non-technical colleagues outside of engineering still used it because it could do complex work like researching and creating a presentation, document, spreadsheet, or tasks that produce rich output. Today, those folks are very heavy users of it.
Being able to work for longer on more complex things drove adoption. OpenAI added the /goal setting to Codex, where you can set up a goal for the agent and it keeps working until it is complete. Between April and May, usage surged from 60% to 90%. Andrew believes improvement in the harness’s handling of long-running tasks was one cause of this:
“The number one thing that is changing is that people are starting to use threads for much longer, and this longer usage has been a breakthrough. It’s surprising to see the sheer length of time that people spend on a thread – even days! They often set a goal and then have the model crank.
Codex being good at longer-running tasks seems to cause people to do fewer things in