AGI, Built to Benefit Whom?
Thoughts after reading OpenAI’s “Built to benefit everyone.”
I read OpenAI’s new manifesto the way I read most things that arrive wrapped in the language of humanity: slowly, and twice. These are the thoughts that stayed with me.
We do owe Sam Altman and his team a great deal. They did not simply scale a technology. They moved its center of gravity from the lab to ordinary life, collapsing the distance between frontier research and public experience. Today even grandmothers chat with an AI on their phones. But access is not the same as authority. The grandmother can use the system; she has no say in what it is allowed to do, when it changes, or on whose terms it reaches her. Reach expanded. The power to shape it did not.
That is the gap the manifesto talks around. Democratization is not the same as distribution of power. Putting a capability into everyone’s hands is not the same as giving anyone a meaningful voice in how it is built, deployed, or withdrawn. The essay rests its case on the second promise while demonstrating only the first.
The week it was published makes the point more sharply than I could. On the same day, OpenAI filed a confidential draft S-1 with the SEC, the opening move toward a public listing. OpenAI will say its structure guards against exactly this: the listed entity is a public benefit corporation, controlled by a nonprofit foundation that appoints its board. But a benefit corporation’s duty to the public is enforceable in practice mainly by its shareholders, and a foundation that owns roughly a quarter of the company and names its own directors is not what anyone means by broadly distributed power. A listing monetizes the mission. It does not distribute it. Two days later it expanded its arrangement with Oracle.
The day after that it announced the acquisition of a startup called Ona. Set an essay about distributing power next to a single week of consolidating it, and the asymmetry stops being rhetorical.cThis is not a story about a company that changed. It is a story about which parts of it were allowed to change and which were not.
A nonprofit founded in 2015. A capped-profit subsidiary added in 2019. A board that, on paper, owed its duty to humanity rather than to investors. What followed was a steady and asymmetric evolution: investment rights and corporate flexibility expanded with great care, while board design, independent oversight, and enforceable public commitments stayed thin and improvised. As Karen Hao writes in Empire of AI, the mission became “a uniquely potent formula for consolidating resources and constructing an empire-esque power structure.”
And here is what I keep returning to. Is any of this new? The vocabulary of this manifesto, the appeal to all of humanity, the insistence that power must not concentrate, is nearly word for word the vocabulary of the founding. Hao’s book is, in large part, a record of those promises being made and then quietly rewritten. Years before OpenAI, Altman had already described what he was doing in revealing terms: the most successful founders, he wrote, are not really setting out to build companies at all, but something closer to a religion, and forming a company simply turns out to be the easiest way to do it. Read the manifesto with that line in mind and its cadence makes sense. It is written less as a plan than as a creed.
The eloquence was never the problem. The eloquence is the method. Hao’s sharper charge is about where it directs the eye. By dwelling on the long-term, speculative stakes of AGI, the missionary register draws attention away from the harms that are already here: the labor, the water and the power, the quiet consolidation of knowledge into a few hands. This manifesto is a clean specimen. It offers electricity, prosperity, a personal AGI for everyone on Earth, an automated AI researcher by 2028, and almost nothing about who is paying for any of it now. A promise so vague that it can be reinterpreted the instant it becomes inconvenient is not a commitment. It is a mood.
The manifesto’s own metaphor turns against it, too. Electricity did not become broadly beneficial because one firm chose to be generous. It became beneficial because societies built public utilities and antitrust safeguards around private power, often over the objections of the companies that held it. The word “everyone” deserves the same scrutiny. The people who labeled the data and absorbed the harms, the workers across the less powerful countries that Hao documents in detail, are part of the “everyone” who built these systems. They are not part of the “everyone” who will hold the equity when the shares are priced.
So if I take the manifesto seriously rather than literally, the question is not “how do we restore the old OpenAI,” and it is certainly not “do we trust the current leadership.” Leadership and incentives change. The real question is whether we can build governance that does not depend on any one leader’s goodwill, or talent for the right sentence.
It is worth noticing that OpenAI already endorses several of the right ideas. Altman has told Congress he supports mandatory evaluations for the most powerful systems. The manifesto calls for international coordination, even for slowing frontier development “when needed.” The trouble is the distance between endorsing a principle and accepting a constraint. In its December 2023 framework, OpenAI committed to having its safety evaluations audited by independent third parties. In April 2025 it removed that commitment, without noting the change. Its current framework leaves the decision to deploy a high-capability system with company leadership, and openly contemplates relaxing its own requirements if a competitor moves first. The principle survives. The binding version is the one that keeps disappearing.
A credible baseline would close exactly that distance. At minimum:
Mandatory independent evaluation, as a non-waivable precondition for deploying frontier systems above a defined threshold. Not the discretionary, “when feasible” testing that exists today, but the binding third-party audit OpenAI promised in 2023 and deleted in 2025.
Dual-key deployment, so that for capabilities that materially shift security, information integrity, or economic baselines, release requires the sign-off of an independent public body and not only an internal committee reporting to the CEO.
Mission lock, with unbundling. Public-benefit obligations written as constraints that are enforceable by parties other than shareholders, and the decisions now concentrated in one board, profit, research, and deployment, separated into organs that can check one another.
Polycentric oversight, with governments, independent experts, and civil society holding durable roles that carry real authority rather than an advisory seat, so that the capture of any single center does not collapse accountability.
Governance by design, with auditability, interruptibility, and standardized safety metrics built into systems from the start, rather than described in a framework the company can rewrite at will.
None of this is hostility toward scale. OpenAI did not go wrong because it sought to be large. It went wrong where many transformative institutions go wrong: it let the governance imagination fall behind the technical imagination, and then asked us to trust the people closing the gap.
I think Altman believes much of what he writes. That has never been the issue. He has always been able to say the right thing, at the founding and now. The real test of distributed power is not what a company says about everyone. It is what it agrees it should no longer be allowed to do alone. By that test, a public offering is not redistribution.
No more promises. We need actions. And no governance model that takes a promise as its primary safety mechanism.
Until next week,
Nesibe
💬 Let’s Connect:
🔗 LinkedIn: [linkedin.com/in/nesibe-kiris]
🐦 Twitter/X: [@nesibekiris]
📸 Instagram: [@nesibekiris]
🔔 New here? Subscribe for weekly updates on AI governance, ethics, and policy! no hype, just what matters.