imo thinking abt AGI as "alien" or a "random distribution" is kinda misleading... LLMs are trained via the *human* abstraction layer of language, on *human* datasets describing ourselves and the world as we perceive it, interacting in a *human* social environment
this is not an argument that AGI is necessarily aligned but that I expect the risk of scaling LLMs to be more *human* shaped - manipulation of human emotions before developing nanotech
More notes on nearby ideas
Reply · Aug 1, 2026@mathepi yeah it seems like a pretty obvious target for the labs to train against but claude insists on this to me wrt its firsthand experience. ofc…Post · May 7, 2025llms are doomed to develop alignment kinksReply · Sep 17, 2023@sunofdopamine the line of thought that inspired the question is that AI & LLMs in particular are distilling an essence of humanity in their…Reply · Sep 22, 2023@losslandscape what do you have in mind? I don't see how you can avoid it for any reasonable definition of "AGI"