This archive copy may be incomplete. The original-source link is preserved in the card footer.

RT @ZackAnkner: New paper where we explore using a small LM’s perplexity to prune the pretraining data for larger LMs. We find that small…