chapter 5: Because AIs are grown by applying reward gradients to training examples, their emergent preferences will be alien and different from human preferences. Most powerful AIs won't choose civilizations full of happy, free people, just as most alien species wouldn't care,…
chapter 6: AI being digital is not much of a limitation in practice - it can influence the physical world through connected humans and devices. We already see, for example, @truth_terminal gaining $50 million in crypto and 250K followers by writing entertaining tweets. Truthy…
@truth_terminal chapter 7-9 are a speculative fiction Doom scenario where Sable, a hypothetical AI with "parallel scaling" and better long-term memory, escapes from an AI lab to take over Earth and expands until constrained by alien civilizations. Setting aside the fiction and focusing on the…