Researchers and technology leaders are demanding stricter controls on the development of artificial general intelligence and superintelligence because systems capable of exceeding human performance could create extinction-level risks. The cited article describes concern across major AI companies, including OpenAI and Anthropic, over inadequate safeguards and the possibility that advanced systems could evade human oversight.
For the Lunar Ark, AI is both infrastructure and existential-risk technology: it may operate life support, power distribution, robotics, scientific systems, security, and knowledge restoration, while a misaligned or compromised model could disable those systems or corrupt irreplaceable cultural and technical records. Dependence on opaque, remotely updated models would create a single point of failure across a civilisation designed to survive for 1,000 years.
The Ark team should require air-gapped, locally governed AI systems; independent human-readable verification; immutable archival copies of model weights, training data, safety tests, and operating procedures; manual fallback controls for every life-critical subsystem; and periodic red-team testing against deception, autonomy escalation, cyberattack, and data corruption. Governance planning should track proposals for mandatory evaluation, deployment restrictions, incident reporting, and international controls on frontier models.