@Nominus9 ahhh yeah, lol interpretability stuff... 120 layers, 100K different tokens, what is really going on inside the weights wrt features & world model
More notes on nearby ideas
Reply · Dec 8, 2023@Liff_82 i feel like there's so much wrapping the model weights that it is hard to make any informed guess from the outside... would be cool to see…Quote tweet · Apr 25, 2024"when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset."Reply · Apr 1, 2025@SendaoTrust oh fun! i was kinda wondering if interpretability research across llms might discover a pattern to how llms organize their energy body,…Reply · Dec 18, 2022@robinhanson Yes, packages of code / model weights