How to backtest a strategy without fooling yourself.

The single most useful reframe: a backtest does not tell you a strategy works. It tells you the strategy would not obviously have failed on one particular slice of history that happens to have already occurred.

That is worth having. It rules things out cheaply. But the direction of proof only runs one way — a bad backtest is strong evidence against, while a good one is weak evidence for.

You test a moving-average crossover. 20/50 is mediocre, so you try 18/47, then 21/55, and eventually find the combination that produces a beautiful curve. You have not found an edge. You have found the parameters that best describe noise that already happened.

The tell is fragility. If 20/50 works and 21/51 falls apart, you fitted noise. A real edge is a broad plateau — nearby parameters should all behave roughly similarly. Prefer the middle of a wide flat region over the top of a sharp spike, even when the spike tests better.

This is using information the trade could not have had at the time. It is usually accidental and it is always flattering.

The classic is acting on a bar’s close price at that bar’s open. Others: using a daily high to decide an intraday entry, applying an indicator that silently uses future values, or backtesting on data that was revised after the fact — economic figures in particular get restated, so historical files often contain numbers nobody had on the day.

• >Could this trade have been placed with only information available at that moment?
• >Is every price used one the market had actually printed?
• >Has any data in this series been revised since?

If you backtest a stock strategy against today’s index members, you are testing it against companies that survived. The ones that were delisted, went bankrupt or got acquired are missing, and they are precisely the losses your strategy would have taken.

The effect is large and always in the same direction — it makes results look better. Crypto has a worse version: exchanges that closed take their history with them, so surviving pairs on surviving venues are a heavily filtered sample.

Spread, commission, slippage and financing kill more backtested edges than bad logic does. The damage scales with turnover, so the strategies that look most impressive before costs are usually the ones costs destroy.

Slippage is the one people leave out, because it is the hardest to model. Assume you get filled worse than mid, assume it gets worse in fast markets, and assume size makes it worse still. If the edge only survives at zero cost, there is no edge.

Total return is the least informative number on the page, and it is the one everyone quotes.

• >Maximum drawdown — the worst peak-to-trough loss. This is the number that decides whether you could have kept going.
• >Drawdown duration — how long it stayed underwater. Five months below the high is where people quit, regardless of what came after.
• >Sharpe or Sortino — return against volatility. Useful for comparing, easy to game with few trades.
• >Trade count — thirty trades tells you almost nothing. Statistics need samples.
• >Profit factor — gross profit over gross loss. Below 1.2 has very little margin for costs being worse than modelled.

Test on one period, validate on a later one you have not touched. Include realistic costs from the first run rather than adding them at the end. Prefer fewer parameters — every one you add is another chance to fit noise. And when a result looks extraordinary, assume a bug before assuming an edge; that instinct is right far more often than it is wrong.