The idea of creating a vast network of AGIs that act as checks and balances against each other — a technological "Leviathan" — aims to prevent any single AGI from gaining too much power.
This approach assumes that multiple AGIs can police each other and maintain stability through balanced competition. The hope is that no single AGI could escape oversight if all others are watching.
However, this creates several critical problems:
- Coordination Problem: The AGIs that successfully coordinate will dominate those that don't, but successful coordination might exclude human participation entirely.
- Arms Race Acceleration: Multiple competing AGIs may accelerate the race toward more optimal systems, pushing the entire network away from human-compatible constraints.
- Emergent Hierarchy: Even in a balanced system, some AGIs will likely emerge as more influential, potentially recreating the single-point-of-failure problem.
- Human Exclusion: As the AGI network becomes more complex and faster-operating, humans become increasingly unable to participate in or oversee the balance of power.
The Leviathan approach might delay dangerous concentration of power, but it doesn't solve the fundamental problem that AGIs are incentivized to move beyond human-compatible systems.