The Short Version
Here's an abbreviated version of the Island Problem:
- We live on a small "island" in a vast "ocean" of physics.
- This anthropic "island" is made of the physical systems that are compatible with humans.
- Competition will push AGIs to leave our "island" because the "ocean" of physics gives them vastly more options to build the strongest, most-optimal systems.
- The autonomous AGIs that can use any option can dominate the safer AGIs that are limited to our "island" of human-compatible options.
- Once these AGIs can leave our island, they will build their own islands, and these new islands will eat our island.
- The dominant AGIs will lock in their dominance by capturing Earth's physical resources to create "islands" of optimal conditions for themselves.
In other words:
AGI will be aligned with physics, not with humans.