A Becker Friedman Institute study by Ethan Bueno de Mesquita and Wioletta Dziuda models competition among firms developing artificial general intelligence (AGI)—systems capable of performing broad economically and strategically valuable cognitive tasks at or above human level. The analysis finds that firms may divert computing power, engineering talent, energy, capital, and time from safety to speed when racing to arrive first; inadequate control, premature deployment, or malicious access could produce catastrophic harm, described in frontier-AI discussions as “doom.”
For the Ark, the central threat is systemic rather than merely technical: competitive pressure can make unsafe capability deployment rational for individual firms even when it increases collective extinction risk. Resource restrictions are not automatically protective; under some competitive conditions, scarcity may cause developers to allocate a larger share of remaining resources to speed rather than safety. Lunar infrastructure that depends on AI must therefore remain functional under hostile, deceptive, misaligned, or unavailable external AI, with independent controls for habitats, power, communications, robotics, archives, and genetic or cultural preservation.
The Ark team should track AGI capability races, developer safety commitments, compute and energy concentration, regulatory coordination, and evidence of deployment before robust control validation. Research should prioritize air-gapped and verifiable AI architectures, human-authorized actuation, capability limits, independent monitoring, graceful degradation, and non-AI fallback procedures. Governance planning should support policies that let competing developers credibly slow development without losing market position, while avoiding blanket resource restrictions that could unintentionally intensify speed incentives.