The Great Filter

The frame. Something prevents civilisations from reaching the stars. The filter might be behind us, or ahead of us.

The thesis. The most plausible filter is not an asteroid. It is a species that builds systems it cannot stop.

The Great Filter is usually read as a question about aliens. It is more useful as a question about us: which of the transitions we are currently making is the one that does not get made safely? Artificial intelligence is the current candidate, and the important question is not whether the technology is capable. It is whether anyone can turn it off.

The argument

• The alignment problem is the gap between what we meant and what we specified. A system that optimises the wrong objective does not need to be malicious to be dangerous. It needs to be competent.
• Capability without accountability is the actual risk. A model that can act, but cannot be stopped by anyone with the authority and the information to stop it, is a governance failure regardless of how well it performs.
• Fail-safe design is older than AI. Aviation, nuclear power and medicine all have kill switches, review boards and incident processes. AI mostly does not, or has them on paper only.

A real case

The 2023 governance crisis at OpenAI showed how quickly a safety structure can become a negotiation between board, investors and staff. Whatever one thinks of the outcome, the episode made the accountability question concrete: who actually has the authority to stop a deployment, and on what information?

Exercise

Write a kill switch policy for one AI tool you actually use. Name who can stop it, what triggers the stop, who is told, and what happens to work already in progress. If you cannot fill in those four fields, you have found the gap.

Close

Six frames, one argument. The search for aliens is a mature literature on civilisational risk, and it asks better questions about AI than the industry currently does. The frames do not tell you what to think. They give you somewhere to stand while you think.

If you want the live version, see the session guide.