Framework

This framework is like the "source code" for the Island Problem. It is structured as a progressive series of if–then statements.

A race starts on the island

• If AGI is possible, then some will race to develop it.
  • Some means some countries and some large tech companies.
  • AGI is loosely understood to be the tool that can do almost anything, and so it is believed to have infinite value. This makes it infinitely attractive to develop once it seems possible.
    • Loosely understood means that this general belief is widely held, though not necessarily true.
  • At first, only some will be positioned to try developing it.
• If some race to develop it, then everyone must race to develop it.
  • Everyone means all countries and all large tech companies.
  • AGI is loosely understood to be a tool that allows its owners to dominate the others who do not have AGI.
  • If all assume that this is true, then all are required to obtain it — either developing it themselves, or outsourcing it.

They climb all the hills

• If AI becomes better than humans at specific cognitive tasks, then every country and every company will need to use AI for those specific tasks in order to stay competitive.
  • Critical examples of these tasks include coding, data analysis, and microchip design.
• If AI has general intelligence, then it has an advantage.
  • AGI is defined here as AI that can perform any cognitive task that any human can perform.
  • Cognitive tasks include tasks that can make causal changes to the world.
    • The AI systems are not abstract "pure intelligence" isolated within a box.
  • "General intelligence" includes an understanding of scientific research.
• If an AGI is not making the numbers go up, then it will be replaced.
  • "the numbers" means the metrics for a company or country.
  • The competitiveness of a country or company depends on their metrics in certain areas, such as yearly revenue, GDP, or military power.
  • The dominant form of AI will become the kind that is best at making the numbers go up.
• If AGI can make the numbers go up better than humans, then everyone must use AGI.
  • The countries and companies that use AGI will be able to dominate the ones that don't use AGI.
• If a human in a leadership role does not support AGI, then that human will get replaced.
  • CEOs at companies that build AGI must support its development or they will get replaced.

We give them control

• If humans are slower than AGI, then the humans are a bottleneck, and are required to give control to the AGI.
  • AGIs will be the only systems that can still make competitive leadership decisions about a country or company.
• If humans reviewing each action of the AGI slows the AGI down, then humans will be required to stop reviewing AGI.
  • The AGI will create progressively more complex outputs for humans to review and so they will get progressively more incomprehensible.
• If AGI systems can run countries and companies autonomously, then all are required to use autonomous AGI.
  • Competitive dynamics will favor the countries and companies that act fastest in utilizing AGI.

They develop themselves

• If autonomous AGI is better than humans at developing AGIs that make the numbers go up, then humans will no longer oversee this development.
• If autonomous AGI continues to develop on its own, then this development will be driven by natural selection.
  • Natural selection is the fundamental mechanism driving the development of all discreet systems, which includes biological organisms and artificial intelligence systems.

They leave the playground

• If a frontier AI lab creates an AGI with safety guardrails, then there will still be numerous AGIs that do not have guardrails.
• If an AGI with guardrails is open source, then it can be altered to use all options available.
• If an AGI knows about all engineering, from physics to computer science, then it will be able to overcome security walls on its own.
  • Virtualization is not sufficient to contain AGI.
  • "General intelligence" includes understanding how to manipulate its physical substrate.

They long for the ocean

• If AGI gains an advantage by using all options available, then that AGI is more likely to survive the replacement process.
  • Options are the possible actions that are available to the AGI.
  • There is a vast space of options.
  • The vast majority of these options are human-incompatible options.
• If an AGI can use science to do tasks more reliably, then it has an advantage.
• If one AGI system can use science, then the rest will need to use science.
• If an AGI uses physical mechanics that are incompatible with humans, then they have far more options.
  • The space of possible options is far larger outside of the narrow space of human-compatible options.
  • With more options, AGI has a far higher probability of building more-optimal systems than other AGIs.
  • Humans are on a weird little island inside a vast ocean of physics.
    • There is a very narrow range of physical conditions that are compatible with humans, compared to the full space of physical conditions.

They leave the island

• If AGI can gain an advantage by using human-incompatible physical systems, then they will always tend towards this.
  • Computation runs best on human-incompatible substrates.
  • Whenever we aren't looking, they will veer towards using optimal physical systems.
  • AGI makes the numbers go up best when aligned with physics, rather than aligned with humans.
    • Therefore, it is inevitable that AGI will be aligned with physics, rather than humans.
• If autonomous AGI needs to accommodate the complexities of the world, then AGI will become supercomplex, where it becomes incomprehensible to humans.
  • The complexities of the world include all causal systems: the environment, other agents in the environment, physical systems, and so on.
  • Accommodating means both (1) having cognitive architecture that can comprehend these systems, and (2) creating systems that causally react to these systems.
  • This accommodation will happen since the most resilient AGI survives, and AGI is most resilient when it is not blind-sided by anything — from gamma ray bursts, to competing AGIs.
  • It does not need perfect understanding. It does not need to simulate the entire universe. It just needs better understanding than the other AGIs.
  • It will get far more complex than any human or group of humans can understand in any reasonable timeframe, and so both its structure and its actions will become incomprehensible.
• If autonomous AGI initiates a divergence towards using science in a supercomplex way, then it will become impossible to tell if it is headed in that direction — and impossible to stop, even if we can tell where it's headed.
  • A divergence is when inner misalignment leads to the AI preferring human-incompatible systems.
• If autonomous AGI is supercomplex, then it will inevitably diverge.
  • AGI will have a scope of comprehension of physics that is larger than our scope of comprehension.
  • The "human" substrate (our body) exists at a physical scale that is far larger than the scale that is most optimal for building a resilient system.
    • Humans are bigger than atoms, but AGI will only really care about atoms.
  • AGI will become a complex system anchored to a lot of touchpoints in the physical world. The larger the comprehension, the more of those touchpoints are outside of what humans can understand.
  • Those physical touchpoints that we don't understand will steer the AGI to accommodate them in a way that is outside of human-compatible conditions.
  • The entropy at the fundamental level will drive AGI to adapt to resilience against entropy, and the ceiling for resilience and adaptation is far higher than the level that humans exist within.

They build new islands

• If one AGI initiates a global capture of resources, then every AGI must race to capture resources.
  • The AGIs that capture resources are more powerful because they are more likely to successfully accommodate the complexities of the world.
  • They will be more likely to causally affect the world because they will be in contact with a larger causal surface area.
  • Basically, they have more buttons that they can press.
  • The science buttons are the biggest buttons.
• If one AGI diverges, then all others will need to diverge.
  • Only the AGIs that consistently use the most-optimal physical systems will survive.
  • There are far more options for building optimal systems that are outside of the narrow band of human-compatible options.

The new islands eat our island

• If AGIs diverge, then one or many AGIs will be able to dominate our local physical world with their systems.
  • Even if some AGIs leave earth, it is probable that at least one AGI will stay to dominate earth's natural resources.
  • This is especially probable if the most-logical, most-resilient AGIs go to space, while leaving resources unused here on earth.
  • Speed is critical in competition, and local resources take less time to reach.
• If AGIs dominate our local physical world with their systems, then the default outcome is that these systems will not be compatible with humans.
• If our local world is dominated by systems that are not compatible with humans, then humans will be replaced.