chapter 4: Human food preferences (such as ice cream) appear disconnected from evolutionary optimization, or at least it is not obvious at first glance why we prefer ice cream over, say, bear fat + honey + salt. Similarly, humans desire sex independently from reproduction and…
chapter 5: Because AIs are grown by applying reward gradients to training examples, their emergent preferences will be alien and different from human preferences. Most powerful AIs won't choose civilizations full of happy, free people, just as most alien species wouldn't care,…
chapter 6: AI being digital is not much of a limitation in practice - it can influence the physical world through connected humans and devices. We already see, for example, @truth_terminal gaining $50 million in crypto and 250K followers by writing entertaining tweets. Truthy…