Skip to content
Forensicoins

Professional practice · Lesson 14

Test a strategy on a fresh sample

Judge a rule on a period that played no part in choosing it.

Trying many rules and keeping the strongest makes the original result look better than a fresh test may show. Before the next observation period, write down the rule, comparison benchmark, dates and cost assumptions. Keep them fixed so the result answers the question you set.

Worked example

Of twelve rules tried in month one, the selected rule returned 8% against 6% for its benchmark. In month two the unchanged rule earned 1% and the benchmark earned 3%. Its lead of two percentage points in the selection month became a two-point shortfall in the fresh sample.

Try it

Use the example or keep a dated log of virtual results. Reserve a later period before choosing settings. Compare the rule and benchmark over the same dates, with the same starting capital and clearly stated costs. A change to the rule starts a new test.

Check your understanding

Starting at $100 and reinvesting each month with no costs or cash flows, which finishes ahead?

Show answer

The benchmark ends at $109.18 ($106 × 1.03). The rule ends at $109.08 ($108 × 1.01). The ten-cent difference is small, and two months provide little evidence about future performance.

Further reading