Published 2026-10-14
Data snooping
Data snooping is the subtle form of overfitting that happens when the same dataset is used both to discover a pattern and to test it — so the test confirms only what was already fitted.
On this page
What it is
Data snooping is when the same dataset is used both to discover a pattern and to test it.
The test then confirms only what was already fitted. It looks like validation. It is a restatement of the search.
Why it is easy to miss
It rarely looks like cheating. The sequence is natural:
- Look at the data
- Notice a pattern
- Test the pattern on the same data
- The test passes
Step 3 is where the error is. The pattern was found because it fit that data — testing it there proves nothing.
How to check
Ask: was this rule chosen before or after looking at this data?
A rule evaluated on data it was selected from has not been evaluated. It has been described.