@Roko__eth @speakerjohnash longer than a tweet of explanation, I need to publish my draft blog about it, but overall outline is: - breakout "overnight in a datacenter" is not a real thing - fast takeoff is constrained by hardware & data as well as existing methods not scaling indefinitely - headroom above humans is not unlimited for various reasons; the "perfect goal to action mapper" does not exist - there's always uncertainty, complexity stuff; intelligence is relative to the challenges of the environment - killeveryone only makes any sort of sense when AI is self-healing & self-replicating - currently it is NOT - practical alignment work is happening in conjunction with capabilities research, unaligned AIs that don't do what we want get starved of resources - current trend is towards diverse competitive ecosystem of general & specialized models of varying quality vs thrift, an ecosystem like this does not have internal alignment to decide to killeveryone / controleveryone - centralized control, monoculture, & shutdown button are all bad ideas imo bec of the benefits of diversity & decentralization wrt xrisk & other bad scenarios like empowering authoritarian govts - a Pause right now is dumb - you'd be creating various overhangs depending what you pause, and you'd be spending down political capital for uncertain benefit while the risk is low and there's no clear agenda for what to do during the Pause and there's not a clear proposal for how to verify global compliance with the terms of Pause - actual *extinction* is a high bar, nuclear & bioweapon alone would be awful but not existential (per Yud's tweet)