Held-Out vs. Train-Time API Fixtures in Agent Evals
Contaminated eval fixtures measure memorization, not whether agents actually learned to use tools.
Dmitri Voloshyn
1 story
Contaminated eval fixtures measure memorization, not whether agents actually learned to use tools.