This archive copy may be incomplete. The original-source link is preserved in the card footer.
RT @doomed_lidar: Some papers on attention scaling! FNet replaces attention with Fourier transforms - https://arxiv.org/abs/2105.03824 Google showin…
More notes on nearby ideas
Repost · Apr 21, 2025RT @niplav_site: Is the attention mechanism in transformers just pure tanha?Reply · Nov 17, 2023@foomagemindset @BasedBeffJezos sama wanted compute to scale & continue training more powerful models, any way to achieve that was means to a 🚀 endTweet · Aug 14, 2024human attention data via eye trackingRepost · Jun 19, 2023RT @omarsar0: Applications of Transformers New survey paper highlighting major applications of Transformers for deep learning tasks. Inclu…