Published 2026-10-14
Backtest overfitting
Backtest overfitting is the specific failure of running many variants and reporting the best one. With enough attempts, a good result appears by chance alone — and looks exactly like a real one.
On this page
What it is
Backtest overfitting is running many variants of a strategy and reporting only the best one.
This is a distinct failure from ordinary overfitting. The rule here may be simple. The problem is the search: with enough attempts, a good result appears by chance alone — and looks exactly like a real one.
The mechanism
If you test 1,000 random rules, a predictable fraction will look excellent — by construction. Reporting the best of 1,000 is not evidence of edge; it is evidence of having tried 1,000 times.
The tell is the number of attempts, and it is almost never disclosed.
How to check
Ask: how many variants were tested before this one was chosen?
If the answer is not available, or if the search was open-ended — “we tuned until it worked” — the reported result is the maximum of an undisclosed search.