← Essays

Essay

Every Reformation Needs a Prince

Four AI laboratories, one method, a founding generation that keeps splitting. Most of the religious-schism reading fails against the record. One part survives.

Ariel Agor24 min read

Listen · Read by Ariel · click any word to jump

0:00 / · loading…

On the evening of Friday, 17 November 2023, the board of a San Francisco nonprofit fired the chief executive of the most valuable startup in the world, and then discovered that it had not thought about Saturday.

By Saturday it was on the phone to Anthropic. The board had approached Dario Amodei about merging the two companies and handing him the combined thing. Reuters reported it within days, and Sutskever's later deposition and Helen Toner's testimony at the Musk trial both describe the discussion. Toner has disputed parts of the account, specifically that she made the call happen and that other directors were behind a merger, while calling it an option worth considering among a set of difficult options.

Amodei declined. He had walked out of OpenAI two years earlier with his sister and most of the safety team, and being offered the parent back inside forty-eight hours is not a thing that happens to reformers very often.

By Monday, 738 of roughly 770 employees had signed a letter threatening to follow Altman to Microsoft unless the board resigned and brought him back. The count had been 505 that morning and kept climbing all day. Satya Nadella had already offered Altman and Greg Brockman somewhere to land, which turned the threat from a gesture into an address.

Among the signatures was Ilya Sutskever's, who had voted to fire Altman on Friday and now put his name to a document demanding that his own vote be reversed. He posted that he deeply regretted his participation. By Wednesday Altman was back, and the board that removed him was gone instead.

Five days. A nonprofit board that had been constructed, deliberately and at some expense, to be able to remove a chief executive in exactly this situation removed him, and the removal did not survive the weekend.

The comparison to a religious schism writes itself at this point, which is the first reason to be careful with it. Four laboratories, one method they all inherited from the same paper, a founding generation that keeps splitting, and public statements of belief about the end of history that read more like creeds than like product roadmaps.

I reached for that comparison inside a minute, which should have been the warning. It has already been written several times over. There are essays mapping Google to Catholic orthodoxy and the breakaways to Protestant traditions, and a small genre treating scaling as a holy war. The lineage-as-schism reading is not a new observation, and I spent the first week of this treating it as mine.

So take it as a hypothesis rather than a conceit, and run it against the record. The sociology of religion is an empirical literature with measurements in it, and it predicts particular things about groups that split over belief. Most of those predictions fail here, and they fail in a way that is more interesting than a clean fit would have been. One survives, and it is the one the analogy is never used for.

A word first about who is in the sample and who is not. The four are Google DeepMind, OpenAI, Anthropic and the company formerly called xAI, which SpaceX acquired in February of this year and renamed SpaceXAI in July.

I chose them because the question I started with was about American research laboratories with public doctrines, and because the four of them are connected by people who moved between them. Meta is American, larger than SpaceXAI, and I held it out deliberately. That exclusion turns out to matter, and it is addressed below rather than hidden.

So is the better objection, which is that the group with shared texts, entry stigma, in-group vocabulary and career pledges is not any single laboratory but the safety and rationalist network that staffs all four of them at once. That is a real alternative and probably a better fit for the model. It is also a different essay. The unit here is the firm, because the firm is what holds the compute.

The received account of the lineage has DeepMind as the mother house, the transformer as the revelation, and OpenAI as the first defection. Demis Hassabis, Shane Legg and Mustafa Suleyman founded DeepMind in London in 2010; Google bought it in 2014; researchers at Google published "Attention Is All You Need" in 2017; and the breakaways proceed from there in a tidy line.

Almost every join in that chain is loose.

The transformer paper came out of Google Brain and Google Research, not DeepMind, and Brain and DeepMind did not merge until April 2023, six years later and five months before the board meeting above. OpenAI was founded in December 2015, two years before the paper existed, so it cannot have been founded over it.

And OpenAI was founded against DeepMind rather than out of it. Musk's own lawsuit describes Altman recruiting him with the pitch of a nonprofit that would act as a counterweight to Google's control of Hassabis. Ilya Sutskever, the closest thing to a scriptural author in the founding group, came from Google Brain, not from London. The first branch of the tree runs the opposite way from the one everybody draws.

Which leaves the question of what actually produces these splits, and there is a body of work on that already, none of it theological.

Steven Klepper spent a career on spinouts in the American automobile industry and in semiconductors, and the pattern he documented is the pattern here: new firms cluster geographically around a small number of unusually successful parents, they are founded disproportionately by high-status insiders who were blocked from strategic influence rather than by marginal employees, and they appear in bursts after governance shocks.

Gompers, Lerner and Scharfstein found the same thing in venture-backed entrepreneurial spawning. Fairchild Semiconductor produced dozens of descendants including Intel, and the eight men who left William Shockley in 1957 to start it were called the traitorous eight by exactly one person, Shockley, and by nobody since.

I set a win condition before looking at the answer, because I wanted to be able to lose. If the spinout literature predicts the lineage, the timing, the geography and the identity of the people who leave, then schism is a decoration on top of labor economics and the essay should stop.

Run it. Amodei was a vice president of research who left over direction. Sutskever was chief scientist who left after losing a governance fight he started. Mira Murati was chief technology officer. Suleyman co-founded DeepMind and sold his next company into Microsoft. These are senior insiders with influence but not control, departing after a shock, into the same three metropolitan areas, with capital waiting. Klepper called that in 1996.

That is where I expected to stop. The condition I set had fired and the interesting version of the essay was dead.

The cleanest case is the one the schism reading tends to skip, because there is no doctrine in it anywhere.

Musk proposed in 2017 that OpenAI convert to a for-profit, and by that summer he and Brockman had agreed a for-profit was the logical next step, but Musk's terms were majority equity, absolute control and the chief executive's chair. The others refused him. He left the board in February 2018, sued six years later, and in May of this year a jury in the Northern District of California took under two hours to find unanimously for OpenAI on the statute of limitations, without reaching the merits at all. He has said he will appeal.

Nothing in that sequence is theological. It is an argument about who runs the thing, held first in a boardroom and then in a courtroom.

So the residual is narrow, and I want to state it narrowly, because a narrow true claim is worth more than a wide vague one. Two things in this story the spinout literature does not predict.

The first is that these firms publish creeds. Fairchild's founders did not write essays about what the world looks like after transistors.

The second is that in November 2023 the money lost for five days. Employee equity, the investors, Microsoft's position, every material interest in the building pointed one way, and a nonprofit board pointed the other, and the fight took most of a week to resolve rather than most of an hour. That is not how a labor market behaves.

The creeds are real and they are specific.

DeepMind ran for years on "solve intelligence, and then use that to solve everything else," which is not a business plan but an eschatology with a work order attached. Amodei's "Machines of Loving Grace" is a description of the good end state, compressed decades of biology and development arriving inside a few years. Altman's "The Gentle Singularity" is the same end state with the temperature lowered and the transition made smooth. Musk's formula is that the thing must be maximally truth-seeking, which locates the danger in dishonesty rather than in capability.

Four accounts of how the world ends and who is fit to bring it. Walter Isaacson reports Larry Page calling Musk a speciesist for preferring humans to whatever comes next, at a birthday party in Napa that Isaacson dates to 2013 and other accounts place in 2015. That is not a disagreement about products.

It is fair to object that these documents are published by companies raising money, and that a sincere laboratory and a cynical one performing sincerity would write identical essays. I cannot tell those two apart from outside the building, and neither can you. That is why the rest of this looks at structure instead.

But notice what the objection concedes. Nobody in semiconductors needed to perform an eschatology to raise a round.

Yann LeCun is the case I left out of the sample on purpose. He founded and ran FAIR, Meta's research laboratory, and is the only principal at that level who rejects the method itself. His position is that large language models are confined to text as a medium, and therefore "can't truly reason or plan, because they lack a model of the world."

He announced his departure from Meta in November 2025 after friction with Alexandr Wang, and launched AMI Labs, Advanced Machine Intelligence, with Alex LeBrun as chief executive and himself as executive chairman, building on video and spatial training rather than text.

That is the only genuine doctrinal schism in the story. Everyone else fights about who holds the keys while agreeing entirely about the text. LeCun rejects the text.

And here is the price of it. AMI has been reported at around three and a half billion dollars. Anthropic raised sixty-five billion in May at a post-money valuation of nine hundred and sixty-five billion. OpenAI was at eight hundred and fifty-two billion in March. Sutskever's Safe Superintelligence, which has shipped no product at all, sits at roughly thirty-two billion after Nvidia put five billion in this July.

The one man who broke over what to believe runs the smallest house in the story by something close to two orders of magnitude, and the man who broke over nothing but refuses to ship is worth ten times what he is.

One more line on this, because it cuts against me. LeCun's own exit quote was that you certainly do not tell a researcher like him what to do, which is a claim about authority, not about creed. Even the purest doctrinal break in the set has a control fight inside it. That is a result, not an embarrassment.

Meta's absence from the sample stops being a convenience at this point and becomes a finding. Meta is the one large American laboratory with no eschatology, no public account of the end state, and no founder-prophet, and it is the one that behaves like a normal corporation, reorganizing its research function and losing its chief scientist to a startup the way normal corporations do. It is the secular state sitting next to the eschatological sects, and it is useful precisely because it shows what the others look like without the creed.

Max Weber sorted legitimate authority into three kinds. Traditional, resting on inherited custom. Legal-rational, resting on office and written rule. Charismatic, resting on the person, on the conviction that this particular individual has something the rules cannot confer.

In November 2023 the OpenAI board held legal-rational authority in its purest form. It had the charter, the votes and the documented power to act. Altman held charisma, a patron and the staff. The contest ran for five days and the office lost.

Weber also said what happens next, and it is not that charisma wins permanently. Charisma is unstable by construction, because it cannot survive the person who carries it, so successful movements routinize it into offices, rules and inheritable structures. A great deal of what these companies have built since 2023 is exactly that process, visible and dated.

Anthropic's answer is the Long-Term Benefit Trust, built around a class of stock called Class T, created when the Series C closed and held only by the Trust. It carries the power to elect and remove directors on a schedule that phases in by time and by fundraising milestones, from one seat to a majority of the board within four years.

It carries almost no money. The shares are few and their economic rights are limited, which means the body accumulating the votes does not accumulate the upside.

OpenAI's answer, completed on 28 October 2025, was to put the operating company inside OpenAI Group PBC underneath the OpenAI Foundation, with the Foundation holding both control and conventional equity valued at around a hundred and thirty billion dollars, after months of review and concessions involving the attorneys general of California and Delaware. Microsoft came out of it with a position valued at about a hundred and thirty-five billion, somewhere between a quarter and a third of the company depending on which conversion basis is quoted.

Two different institutional answers to the same question, which is how to stop the next November from being decided by whoever the staff likes best. A trust holding votes that pay nothing. A foundation sitting on top of a corporation with two state regulators watching. Both are attempts to convert a founder's conviction into something that outlives him, which is the oldest problem in the sociology of religion and one of the few places where the comparison pays.

Albert Hirschman supplies the other. In 1970 he distinguished exit, leaving, from voice, staying and complaining, and argued that voice gets its force from the credible availability of exit.

The letter in November 2023 is a textbook case. Seven hundred and thirty-eight people did not leave. They demonstrated that they could, to a named destination, at once, and that was sufficient. Exit was never exercised, which is why it worked.

Strictness is where the analogy gets pushed hardest, and where I had it wrong.

The relevant literature is precise about what it measures. Laurence Iannaccone's work on why strict churches are strong, and Richard Sosis and Eric Bressler's study of nineteenth-century American communes, are about the costs of membership. Prohibitions, sacrifices, stigma, surrendered outside options. The mechanism is screening. Demands that are expensive to meet drive off people who are not committed, which raises the average commitment of everyone left and makes collective goods possible.

Sosis and Bressler found that religious communes outlived secular ones, and that costly requirements predicted longevity only in the religious ones. What none of this measures is what it costs to leave. That is a different variable, and I spent a full draft conflating the two, which makes the whole apparatus say things it does not say.

Applied properly, the AI laboratories do impose real membership costs. Publication restrictions on people who came out of a culture that publishes. Compute rationed by internal allocation. Non-disclosure obligations that constrain where a researcher can go next.

And the strongest case, which cuts against the easy version of this argument and is better for it.

When Daniel Kokotajlo left OpenAI in 2024, he refused to sign the non-disparagement clause in the standard exit paperwork, and in doing so gave up vested equity reported at somewhere between one and a half and two million dollars, which he has described as the large majority of his family's net worth. He and his wife spent two months with lawyers before deciding. That is a costly signal in the strict technical sense. Expensive, voluntary, hard to fake, and correlated with actually believing the thing.

Then the complication. When the clause became public in May 2024, immediately after Sutskever and Jan Leike left, OpenAI retracted it. Altman said that vested equity is vested equity, full stop, and that it was one of the few times he had been genuinely embarrassed running the company. The company released former employees from the obligation on 24 May and equalised secondary share sales in June. Kokotajlo kept the money.

Two separate things are true in that. The sacrifice was real, and it was refunded.

And the clause itself was never a membership cost at all. It was an exit penalty, a device for suppressing departure after the fact rather than screening commitment on the way in, which is why it collapsed the moment it was visible. Strict groups do use exit penalties, shunning and expulsion among them, so this is not a clean contradiction of the model. But it is not the mechanism Iannaccone measured either, and the one clear case of a researcher paying a genuine price for conscience ends with the price being returned and the practice abolished.

One house did take something like a vow, and it is the strongest case the ascetic reading has. Ilya Sutskever left in May 2024 and announced Safe Superintelligence the following month with a single stated product and no intention of releasing anything before it. No revenue, no interim models, no commercial distraction, roughly fifty people. In the language of the literature that is a high-demand group with an unusually pure prohibition, and it is the closest anyone in this story comes to walking into a desert.

It is valued at about thirty-two billion dollars. Nvidia put five billion of that in this July.

The company that sells the land has taken a position in the hermitage, and the hermitage has raised eight billion dollars without shipping a line of product to anyone. Renunciation was not punished here. It was underwritten, at a valuation ten times what the only doctrinal dissenter in the story commands, because a credible claim on superintelligence is worth more to a patron than a product is.

So the ascetic reading does not survive. These are not people walking into a desert.

The finding has been sitting in the history the whole time. No reformation in the record ran without a prince.

Luther survived his excommunication because Frederick the Wise staged a kidnapping and hid him at the Wartburg, where he translated the New Testament into German at a nobleman's expense. The English Reformation ran on the Crown and on the proceeds of dissolved monasteries, which is to say on confiscated land.

The Peace of Augsburg in 1555 settled eighty years of religious war with the principle cuius regio, eius religio, whose realm, his religion, and in doing so conceded that the unit of religious decision in Europe was the territorial prince. And Cardinal Richelieu, a prince of the Church, funded Protestant armies for thirty years because weakening the Habsburgs mattered more to France than the confessional question did.

Doctrine travels on somebody else's money, always, and the shape of a reformation is set by who is paying and what they want out of it.

OpenAI worked this out earlier than the others and wrote it into its capital structure. Having refused Musk's terms in 2017, it still needed the money those terms would have bought, and in March 2019 it built a capped-profit vehicle, OpenAI LP, with investor returns limited at first to a hundred times the money in and everything above that reverting to the nonprofit. In July 2019 Microsoft put in a billion dollars and took exclusive rights to sell the resulting software through Azure.

Four years before the board meeting, the arrangement that would decide the board meeting was already signed. Luther did not defeat the Emperor at Worms. Frederick did, by owning the road home.

Compute is the land. It is the one input that cannot be improvised, cannot be raised from congregants, and is controlled by a very small number of parties who are not themselves laboratories.

Microsoft holds the largest outside position in OpenAI Group. Microsoft also agreed in November 2025 to invest up to five billion dollars in Anthropic, alongside Nvidia at up to ten billion, with Anthropic committing to purchase thirty billion dollars of Azure compute in the other direction. Amazon and Google are both substantial Anthropic investors.

SpaceXAI resolved the question in the most direct way available by having its founder's other company acquire it outright, which is less Henry VIII making himself supreme head of the English church than it is the same instinct with better lawyers: if the authority is going to be contested, own it.

One prince, Microsoft, is now funding two rival churches while holding a quarter of one of them. That is Richelieu exactly, and it is not hypocrisy. It is what patrons do. A patron backs the outcome that leaves it stronger, and a world with two dependent laboratories is better for the patron than a world with one independent one.

This is also the test the stricter house has actually faced. In February the Secretary of Defense designated Anthropic a supply chain risk to national security, a label previously used for firms tied to foreign adversaries, after the company refused Pentagon terms that would have permitted use for all lawful purposes and held lines on lethal autonomous weapons and mass domestic surveillance.

Anthropic went to court. In August, Judge Rita Lin in the Northern District of California vacated the designation in a fifty-nine page summary judgment on First Amendment retaliation, due process and administrative law grounds.

The record is not a clean victory, and I would be selling you something if I told it as one. Anthropic lost its bid for emergency relief in the D.C. Circuit in April, and the government's appeal of the earlier injunction is held in abeyance. But the company did refuse revenue on a stated principle, absorb a federal designation for it, and litigate for six months, while raising sixty-five billion dollars in the middle of the fight.

Not a monastery. A well-capitalised institution with a public creed, a patron structure, and a legal department, doing the thing the creed said it would do, at a price it could comfortably afford.

The Cistercians who broke from Cluny in 1098 because Cluny had grown rich were, within a century, efficient farmers and major wool traders who had to be reformed in their turn. Nobody had to abandon a belief for that to happen. The belief was sincere and the endowment grew anyway, and those two facts have never been in tension outside of sermons.

The question I started with was whether these breakaways are searching for a better operating system for large-scale human cooperation, and it deserves a direct answer rather than a citation fight about group selection.

An operating system for cooperation is a set of rules that makes people behave well toward strangers when nobody is checking, enforced by something other than a contract. Religions do it with belief, ritual and the threat of exclusion. Firms do it with pay and monitoring.

What is genuinely new in these four is that they are trying to do it with corporate structure: a share class with votes and no money, a nonprofit holding a controlling stake, a published policy that commits the company to stop at defined capability thresholds. That is an attempt to write conscience into a cap table.

Whether it holds is the only question about these companies that will still matter in ten years, and it is a governance question, not a theological one. I think it mostly will not hold, and I would rather be wrong about that than right.

Against the win condition I set, the lineage reading lost and the strictness reading lost. So should the costume version, where each company is assigned a historical analogue and wears it for a chapter, because with four firms and four stock comparisons somebody gets flattered by nothing more than arrangement.

Weber, Hirschman and the princes survived. Charisma beat office in November 2023, and both surviving structures are attempts to make sure it cannot happen twice. The threat of exit worked as voice because the exit was credible and the destination had a name. And the reformation is running on patronage, as every reformation has, with the patrons holding the land and funding both sides of it.

That last one carries a prediction, which is the only thing a frame like this is good for. If these are patron-backed reformations rather than ascetic sects, then restraint holds exactly as long as the patron finds restraint useful, and the informative moment is the first time it stops being.

Here is what to watch for, and you can check every item yourself from public filings and company blogs. A capability threshold in a published safety policy that gets revised rather than triggered. A refusal of revenue that is quietly not repeated once a competitor takes the contract. A resignation on principle that produces no letter. A trust or a foundation whose composition is adjusted in the year before a listing.

Any of those is the endowment winning.

The opposite counts as evidence too, and hold it to the same standard. A shipped product delayed past a quarter for a stated safety reason. A company lobbying against a rule that would have helped it. A governance body outvoting its own founders once.

Nothing in the creeds will settle it, because the creeds are written to be unfalsifiable and the people writing them mean it anyway. The structure will settle it. Who can remove whom, on what vote, and what happens to the money when they try.

In November 2023 the answer to that question was that the board could remove the chief executive and the chief executive could remove the board, and the second turned out to be faster by a factor of about a hundred. Everything built since, the shares that vote and do not pay, the foundation stacked on the corporation, the attorneys general in the file, is an attempt to give a different answer next time.

The next time is coming, on a schedule set by two confidential S-1 filings and whoever is holding the compute when they price. The princes are already in the room. They have been the whole time, and they are the ones with the land.

Sources

Want this kind of automation working for your business?

Agor AI designs and ships the systems these posts describe, scoped in weeks, not quarters.

Book a Free Strategy Call