Out-of-sample test

What it is

An out-of-sample test checks a rule on data it was not built on.

The rule is developed on one period (in-sample), then evaluated on a period it has never seen (out-of-sample).

Why it is the honest step

Any rule can be made to fit the data it was tuned on. Out-of-sample is the only test that asks whether it generalises — and it is the test that most reported results quietly omit.

Not evidence	Evidence
“It works on the data we built it from”	“It works on data it had not seen”
A statement about fit	A statement about the future

How to check

Ask one question: was this result produced on data the rule was tuned on?

If yes — or if the question cannot be answered — the result tells you about the fitting, not about the rule. See overfitting and data snooping.