How to backtest a trading idea without fooling yourself

Backtesting means testing a trading rule on past data to see how it would have done. It's the best tool we have for weeding out bad ideas before they cost money. It's also very easy to get wrong in ways that make a bad idea look good.

We test a lot of ideas: our own, popular indicators, and patterns people send us. Most of them don't survive. Here are the mistakes that most often make them look better than they are.

1. Look-ahead bias

This is using information in the test that you wouldn't have had at the time. It's often subtle:

• Using a candle's close to decide to enter at that same candle's open.
• Using the daily high or low before the day is over.
• Joining data from a higher timeframe without waiting for that candle to close. An hourly signal that uses the current four-hour candle is peeking at the future.

We once found that a promising gold scalping signal lost most of its edge once the higher-timeframe data was properly delayed. If a result looks too good, look-ahead bias is the first thing to check.

2. Ignoring costs

Every trade pays the spread, often a commission and sometimes slippage. A strategy that makes a tenth of a pip on average before costs loses money after them. Short-term strategies with small targets are hit hardest. Always test with realistic costs for your broker and your trading hours, and then check what happens if they're a bit higher.

3. Unrealistic fills inside a bar

If your stop and target are both inside the same candle's range, the data doesn't tell you which was hit first. Assuming the target was hit first flatters the results, and on short timeframes it can flip a strategy from losing to winning. Resolve those cases on a lower timeframe or with tick data, or assume the worst.

Trailing stops have the same problem. Bar data assumes price moves in a tidy order, which tends to make trailing exits look better than they are in live trading.

4. Curve fitting

Adjust enough settings and any strategy can be made to look brilliant on past data. The more parameters you tune, the more likely you're fitting noise. Warning signs:

• Results that collapse when a setting changes slightly, such as a 14-period indicator working and a 15-period one not.
• Many rules and filters, each added to remove a few losing trades.
• Great results on one symbol and nothing on similar ones.

The fix is to split your data. Develop and tune on one period (in-sample), then test once, without changes, on a period you haven't touched (out-of-sample). If it fails out-of-sample, it's not ready, however good the in-sample results look.

5. Testing too many ideas

If you test a hundred random rules, a few will look good by pure chance. We once tested every indicator in a large library, and hundreds of combinations of them, on gold. None of them beat what chance alone would produce once we accounted for how many we had tried. The more things you test, the higher the bar for any single result.

6. No control test

This is the one most traders skip. Compare your strategy with a control: random entries at similar times, in the same direction, with the same stop and target rules. If your signal doesn't clearly beat random entries, the profit is coming from something else, such as the trend in the period you tested.

Make the control fair. We once compared a breakout rule with random entries during the same day and the result looked like an edge. When we matched the control's entry prices as well, the edge disappeared. The "signal" was simply entering at a different kind of price, and that alone explained the result.

7. Too few trades

Thirty trades can't separate a real edge from luck. Aim for hundreds, across different years and market conditions. Look at each year separately. A strategy that made all its money in one year has one good year, not an edge.

A testing routine

1. Write the rules down precisely before you look at results.
2. Test with realistic costs and without look-ahead.
3. Compare with a fair random control.
4. Tune on one period, confirm once on another.
5. Check every year and several symbols.
6. Paper or demo trade it for a few months before risking real money.

It's slower than optimising until the equity curve looks smooth. But it's how you find out an idea doesn't work while it's still free. If you're evaluating someone else's system, see how to judge an Expert Advisor.

Risk note: trading forex carries a high level of risk and may not be suitable for everyone. This article is for education only and is not financial advice.