Connecting the Dots With a Backtester
A backtester can flatter a strategy that then fails in live trading. Six checks, from data quality to curve fitting, that decide whether a result is worth trusting.
Backtesting a trading idea against historical data, once it has been translated into code, gives deep insight into how the idea has performed. If something worked well in the past, it is likely to work well in the future too. That is conditional, though, and it is common to see backtest results that do not tally with live trading results. Where, then, does the problem lie, and what do we fix?
The answer is often simple: if the end did not meet the expectation, check the means to that end. Reliable backtesting is an involved process that needs a grasp of the real-world market, data feeds, the backtester engine’s capabilities and algorithmic coding. If we were to draw the broad contours of backtesting, these are the points to connect.
Six things to check before trusting a backtest
- Test across a variety of market conditions. Use bull, bear, volatile and flat markets, across different instruments and over a long period. A small sample cannot test how the system behaves in the range of conditions a live market will present.
- Do not over-optimise and curve fit to the peak of profitability. A slight change in conditions, which is inevitable in live trading, will push such a system off the cliff. A system sitting on a plateau of fairly decent profitability is likely to be far more reliable over the long run than one hanging off a cliff.
- Know the backtester’s limitations. Some backtesters confirm a condition on the current bar and trade on the next bar’s open; some are more advanced. For any candle, the backtester cannot know the intra-bar movement, only the OHLC values, so it has to assume how price moved inside the bar. Some support margin checks, some do not. Watch the trade count and the maximum drawdown too. Ideally the backtester trades the strategy as the live market would; a limited one often throws up false results with little correlation to what will happen live.
- Data integrity matters. Splits and bonuses have to be accounted for, and data holes avoided. Real-time data fetched during the session can also change after market close because of feed issues.
- Factor in liquidity, circuit freezes and transaction costs. A fill the backtest assumes is often not achievable in size, and costs are easy to leave out.
- Ask whether you can execute it with computer-like discipline. If the rules are too complex for you to follow in live trading, the live results are likely to deviate significantly from the backtest.
