Intro: Current AI extinction risk is more severe than mainstream AI safety discourse acknowledges. Superintelligence built with current methods will kill everyone on Earth. Superintelligence can still be prevented through coordinated human action.
chapter 1: Humanity's special power, which has enabled us to dominate the Earth and travel beyond it, is our flexible general intelligence. Intelligence consists of prediction (anticipating outcomes) and steering (finding actions to achieve goals). Steering success depends on…
chapter 2: Modern AIs are grown through gradient descent, not crafted by human design. The process involves trillions of parameters that no human understands, just as human DNA consists of billions of base pairs that no human understands. AI development is more like biology than…