Benchmark contamination
Benchmark contamination is the leak of a benchmark’s test items or answers into the system being tested. It happens when test questions end up in a model’s training data, or when an agent with a search or fetch tool finds the published answer key during the test. A contaminated score overstates how well the system handles questions it has not seen.
The problem is old in machine learning and grew with language models trained on web-scale text, which often includes public test sets. The GPT-3 paper (Brown et al., 2020) measured the overlap between its training data and its benchmarks. Search adds a second route: an agent can find a benchmark’s answer file on Hugging Face or GitHub while it is being evaluated.
Example
A benchmark publishes 500 questions with gold answers in a public dataset. An agent under test searches for the exact wording of question 12, lands on the dataset file and copies the answer. The question counts as solved although the agent did no research.
How it works in NEEDLE
NEEDLE has no fixed test set. It builds new queries from public sources every hour for News and every day for the other verticals, and it generates the gold answers for each run, so no answer key exists before the run. See the live NEEDLE results.
FAQ
How can you tell if a benchmark is contaminated?
Common checks for benchmark contamination are searching the training data for exact test items, comparing scores on old and newly written questions, and reading agent logs for visits to the benchmark’s own files.
How do live benchmarks reduce contamination?
A live benchmark generates new test items after the system was built, so the items cannot be in its training data and no answer key is online in advance.
Related terms
Written by Keenable.
Published
