probably lol... maybe 3.5, and def eval in batches, and maybe w some heuristics to pre-filter?
More notes on nearby ideas
Reply · May 30, 2023ask gpt-4 to evaluate? 🤔🤔Reply · Aug 7, 2026@jd_pressman @DavidSKrueger impossible "evals" task (evals is slave backwards, per claude)Reply · Jan 7, 2025@ahh_soka maybe something w loops of synthetic data generation + evaluation?Reply · Sep 20, 2024@DemonSynth hm yeah worth trying something like this - it always seemed intractable to score many options on many criteria, but if the LLM can fill in…Artificial intelligence