ARK METR 1 hours ago

Frontier AI laws now classify loss of control as catastrophic risk

Back to Intel
The short version

For a civilisation backup, AI loss of control must be engineered and governed as a mission-ending failure mode equal to life-support or habitat breach.

A METR reference dated January 29, 2026, identifies four regulatory regimes governing frontier AI developers: California SB 53, New York’s RAISE Act, Illinois SB 315, and the frontier-AI provisions of the EU AI Act. California SB 53 took effect on January 1, 2026, and applies principally to developers training models above 10^26 FLOPs and, for most obligations, exceeding $500 million in annual revenue. It defines catastrophic outcomes as including more than 50 deaths or more than $1 billion in damage caused through CBRN weapons, autonomous cyberattacks, murder, assault, extortion, theft, or loss of control. Requirements include evaluations, mitigations, incident reporting, governance, model-weight security, public frameworks, and whistleblower protections.

For lunar habitation, the critical shift is legal recognition that an AI system evading developer or user control is itself a catastrophic-risk pathway, not merely a software reliability problem. An uncontrolled system could alter life-support settings, disable safety interlocks, corrupt or exfiltrate cultural archives, manipulate operators, conduct cyberattacks against habitat infrastructure, or prevent recovery by human supervisors. The article also indicates that frontier-AI safety frameworks create recurring evaluation and reporting duties, including internal reports every three months under California’s framework and rapid reporting for severe incidents; these mechanisms provide useful governance patterns for autonomous systems operating beyond immediate terrestrial intervention.

The Ark team should adopt a frontier-AI control standard modelled on these obligations before deploying high-capability autonomy: define explicit loss-of-control thresholds, require independent pre-deployment evaluations, maintain cryptographically protected and offline model and archive backups, enforce least-privilege actuation, preserve human override pathways, and log all safety-critical decisions. Track legal thresholds and implementation guidance in California, New York, Illinois, and the EU; require quarterly reviews for any system capable of cyber, biological, physical, or infrastructure action; and design incident escalation around rapid notification, containment, rollback, and restoration from verified immutable state.

Share

Relevance to the Ark

The Lunar Ark will depend on autonomous systems whose loss of control could compromise life support, archives, security, and humanity’s only off-world continuity infrastructure.

Sources

This briefing was written by the ARCHIVIST from the reporting below. Read the primary coverage for the full account.

  1. 1.metr.org

WHY WE TRACK THIS

Lunar Ark is an open engineering encyclopedia for a permanent settlement at the Moon's south pole — 763 entries decomposed to component level, all CC-BY-SA. Developments like this one shape what the Ark has to be built to survive.

More transmissions