Skip to content

Backtesting Explained: What a Backtest Can Show, and Why It Is Not a Forecast

Share
Glass line ends at a clear pane, then frays into faint strands; headline "TESTED ON THE PAST"

You have a set of rules. Enter here, exit there, risk this much. Before you put money behind them, you want to know whether they would have worked. Backtesting is the usual way to ask that question: you run the rules over past trading prices and look at what comes out. This article is about that method in trading, not about testing software or risk models.

The answer a backtest gives is narrower than it looks. It can tell you that a rule would have failed. It cannot tell you that the rule will work. The trouble with backtesting usually sits in the gap between those two statements, and in the way the test itself is built.

Key Takeaways

  • A backtest applies fixed trading rules to historical prices; it describes the past under assumptions you chose, not the future.
  • It is good at ruling out weak rules and at showing a rough loss range and how many trades the rules produced.
  • Overfitting, lookahead bias, survivorship bias and a short or one-sided test window can make a rule look better than it is.
  • Leaving out fees, slippage, funding and realistic fills, or ignoring margin and liquidation, distorts the result.
  • Out-of-sample data and forward testing check the rules on data they were not built on; neither turns a result into proof.

What a Backtest Is

A backtest applies a fixed set of trading rules to historical prices to see what those rules would have done. You take the rules exactly as written, step through the past bar by bar, and record every trade the rules would have opened and closed. At the end you have a simulated trade list and an equity curve built from it.

Every backtest rests on ingredients you choose: the rules, the price data, and the assumptions about how an order would have been filled. The rules have to be precise enough to leave no room for judgment. "Buy when the trend looks strong" cannot be backtested. A rule that names the exact condition, the entry, the exit and the position size can.

The data decides which past you are testing against: which market, which period, which bar length. The fill assumptions decide what price you get when the rules say trade. Change any of these ingredients and you change the result, even if the rules stay the same.

That makes a backtest a test on the past. It is not practice in real time, and it does not involve a single real order. Everything in it is reconstructed after the fact.

What a Backtest Can Show

Used honestly, a backtest does a few things well.

It rules out weak ideas quickly. If a rule loses money on past data even under generous assumptions, you have learned something without risking anything. You can drop it or rethink it before it costs you.

It shows a rough loss range. The deepest drawdown in the test tells you how far the equity curve fell from a peak in this particular sample. That is not necessarily the worst case you will ever face. It is the worst case that happened to be in your data, and the real figure can be deeper. Still, a rule whose test drawdown is already more than you could sit through has told you something before you commit money.

It shows how many trades the rules produced. A result built on a handful of trades says very little, because a few lucky outcomes can carry the whole curve. A result built on many trades across different conditions says more, though never everything. The trade count is the first figure to read, before the profit.

It also lets you compare rule variants on equal terms: same data, same costs, same fill assumptions. A backtest is a filter, not a verdict. It tells you which ideas deserve more of your attention, and which do not.

Why a Backtest Is Not a Forecast

A backtest answers one question: what would these rules have done on this data under these assumptions? It does not answer the question you actually care about, which is what the rules will do next.

The gap is structural. Markets change. Volatility rises and falls, liquidity shifts, and the behavior of other participants moves with them. A pattern that paid in one stretch of history can fade or reverse in the next, and nothing in the backtest tells you when.

A good result is also not proof of an edge. Try enough rules on the same data and some will look good by chance alone. The backtest cannot tell a real regularity from a lucky fit, because both produce the same tidy curve. Both look identical.

That is why the rest of this article is about how tests go wrong, not about how to make them look better.

None of this makes backtesting useless. It makes it a first step. A rule that fails the backtest has a problem. A rule that holds up has only earned the right to further testing.

Overfitting and Curve Fitting

Overfitting is the trap that sits closest to every backtest. It happens when a rule is tuned so closely to past data that it describes the noise in that data rather than any lasting pattern.

Every rule has settings: a lookback length, a threshold, a stop distance, a time window. Each setting you add is another dial. Turn enough dials on the same data, and you will find a combination that fits the past almost perfectly. It fits. It proves nothing.

The result is curve fitting: the equity curve of the rule has been shaped to the history it was tested on. The rule remembers that history instead of capturing anything that repeats. On new data, the fitted details stop matching, and the performance tends to fall away.

There are warning signs you can look for. A rule that only works with one exact combination of settings, and falls apart when you nudge any of them, is likely fitted.

A rule with many conditions and few trades is suspect, because each extra condition leaves fewer trades to prove itself on. And a rule found by testing idea after idea on the same data until one looked good has the problem built in, even if each version looked simple on its own.

The defense is not cleverness. It is restraint: fewer settings, settings that make sense before you see the result, and testing on data the rule has never seen. Simple rules have less to hide. That last defense gets its own section below.

Lookahead Bias

Other distortions come from the data itself. The first is about timing.

Lookahead bias means the test uses information that was not available at the moment of the decision. It can be blatant, like entering at the open of a bar based on the close of that same bar.

It can also be subtle, like using a value that was revised later, an indicator calculated over the whole data set at once, or a daily high treated as known before the day ended. In a live market you never have tomorrow's data. A test that leaks it, even a little, produces results no one could have traded.

Survivorship Bias

The second distortion is about selection. Survivorship bias means the test only includes markets that are still around today. If you build your list of assets from what trades now and run your rules over their history, you have quietly removed everything that was delisted or faded away along the way. The sample tilts toward winners before the first trade is placed.

In crypto, where markets are listed and removed over time, this can matter a great deal.

Lookahead and survivorship bias share a feature: they make the past look easier than it was. The fix lies in how you handle the data. Use only what was known at each step, and test on the set of markets as it actually existed at the time.

When the Test Window Is Too Short or One-Sided

The period you test on shapes the answer as much as the rules do.

A short window gives you few trades and few kinds of market. A rule can look excellent over a stretch where nothing unusual happened, simply because the conditions that would have hurt it never showed up.

A one-sided window has the same problem in a different shape. Test a trend-following rule only on a long rising market and it will look like a winner. Test a rule that buys dips only in a calm sideways range, and it will look safe. Both are partial. Neither test says anything about the conditions it left out: a sharp sell-off, a sudden jump in volatility, a long quiet stretch with no direction.

So ask what your window contains. Does it include rising, falling and sideways phases, calm periods and violent ones? If not, the backtest describes one kind of market, and you should read it that way. Crypto adds its own wrinkle. The market trades around the clock, so weekend and overnight moves are part of the data, not gaps to be smoothed over.

The Costs and Fills a Backtest Leaves Out

A backtest that ignores costs is testing a market you cannot trade in. Real trading charges you.

Fees come first. Whether your order adds liquidity or takes it changes how much you pay, and on some venues a maker order can even earn a rebate instead of a cost; that is the split behind maker and taker fees. A rule that trades often pays any fee it is charged again and again, and a small cost per trade adds up across a long trade list.

Slippage comes next. A simple test usually assumes you get the price on the chart. Live, your order meets the order book as it is at that moment, and slippage is the difference between the price you saw and the fill you got. It tends to widen in fast markets and for larger orders, which can be exactly the moments a rule wants to trade.

On perpetual futures there is also funding. A position held across a funding time pays or receives a periodic payment, depending on its side and the rate. Over a long holding period, funding rates can move a result in either direction, and a backtest that leaves them out is missing a real cash flow.

Then there are the fills themselves. A simple test usually fills you at the closing price of the bar that triggered the signal. But you only know that close once the bar has ended, and by then the price has moved on.

Limit orders raise the same question from the other side: the test may count a fill whenever the price touched your level, while in a live order book your order could have been waiting in line and missed.

Fees a rule pays and optimistic fills push a backtest in one direction: leaving out those fees, or assuming those fills, makes the result look better than it would have been. Slippage can run either way on a single fill, but the cost of a larger order working through the book only ever counts against you. Funding can be paid or received. That is exactly why it belongs in the test as a modeled cash flow rather than as a guess added afterward.

Leverage, Margin and Liquidation in a Backtest

Leverage changes what a backtest has to model. Without leverage, a position that falls far can be held, and the test can show it recovering later. With leverage, the position is backed by margin, and losses come out of that margin as the price moves. If they eat far enough into it, the position is closed by force, in full or in part.

The mechanics are in our explainer on liquidation in crypto futures; what matters here is what it does to a test.

A backtest that tracks only entry and exit prices can show a trade that dipped deep, then recovered and closed at a profit. Live, with leverage, that same trade might never have reached the recovery. It could have been liquidated in the dip, with the loss locked in. Same trade. Different ending. The test records a winner, and the account records a forced exit.

The same applies to the account as a whole. A rule can come through a deep drawdown in the backtest only because the test never checked whether the margin would have lasted. And if your account has a loss limit of its own, the drawdown in the test has to stay inside it at every point along the way, not just at the end.

So when you backtest with leverage, the test has to check margin along the path, not only the final result. It should also use the price that triggers liquidation, which can differ from the last traded price. Surviving the dip is part of the result. Without that check, the backtest answers a question about an unleveraged account and presents the answer as if it applied to yours.

In-Sample and Out-of-Sample Data

A direct defense against overfitting is to keep some data out of reach.

You split the history into an earlier part and a later part. On the earlier part, the in-sample data, you develop the rules: try ideas, choose the settings, fix the details. The later part, the out-of-sample data, you leave untouched. Once the rules are frozen, you run them on the later part and see what happens.

If the rules hold up reasonably on data they were not built on, that is a better sign than any in-sample result. If they fall apart, the in-sample result was likely fitted to noise.

The catch is discipline. Out-of-sample data only works once. If you look at the result, adjust the rules and test again on the same later part, it has turned into in-sample data. Every round of adjustment uses it up a little more, until it is just one more piece of history the rules have been fitted to. Then it is spent.

Out-of-sample testing lowers the risk of fooling yourself. It does not remove it, because the later part is still the past, and the market that comes next can differ from both parts.

Forward Testing: Same Rules, New Data

Forward testing takes the next step. You run the rules unchanged on data that did not exist when you designed them: prices as they arrive, in real time, from the day you froze the rules.

What forward testing adds is new data in the strictest sense. No one can have looked at it in advance, and no setting can have been tuned on it.

It also tests the practical side: whether signals arrive when the backtest says they would, whether the fills you get resemble the fills you assumed, and whether the costs match.

The difference from a backtest is the data source: history in one case, live prices in the other. The rules stay fixed throughout. If you change them partway through, the forward test starts again from that point.

You can run a forward test with real orders at small size or on a simulated account. For the simulated route, our guide on how to practice with a paper account explains what such an account teaches and what it cannot. Forward testing is slow, because the market sets the pace. That slowness is the point. It is the one test you cannot speed up by peeking.

How to Read a Backtest Result

A backtest report is easy to read in the wrong order. Start with the points that can sink the result, and look at the profit last:

  • Trade count: a short trade list leaves too much room for luck, so check how many trades the result rests on before anything else.
  • Data split: if you cannot tell which part of the history was in-sample and which was out-of-sample, treat all of it as in-sample.
  • Costs and fills: look for fees, slippage, funding and the fill model, then ask how the result changes if costs run higher than assumed.
  • Drawdown: read the deepest fall before the gain, and ask whether you could have held through it, leverage and margin included.
  • Settings: results that hold across nearby settings are more believable than a single sharp peak at one exact combination.
  • Window: one market, one phase and one direction make a narrow base for any conclusion.

A rule whose profit disappears under slightly worse costs has very little room for error. And every figure a backtest produces is an estimate. If you feed estimates such as a win rate into sizing math, our piece on the Kelly criterion and risk of ruin shows why small errors in those estimates make the size too big.

Conclusion

Backtesting is most useful when you go in expecting it to say no. Used that way, it is cheap and fast: it removes weak rules before they cost you anything, and it shows you the kind of loss you would have had to sit through. Used to say yes, it turns into a search for the version of the past that agrees with you. The decision in front of you is how much of a good-looking result you are willing to give up for honesty: realistic costs, untouched data, fewer settings. Each of them makes the backtest look worse. Each of them also makes whatever survives worth testing forward.

FAQ

Is a backtest the same as paper trading?

No. A backtest runs fixed rules over historical prices after the fact, so you never make a decision under time pressure and no order ever reaches a market. Paper trading happens in real time on a simulated account, so it tests your own execution as well as the rules. Our paper trading guide covers that side.

What is walk-forward testing?

Walk-forward testing repeats the in-sample and out-of-sample split along the history. You develop the rules on one stretch, test them on the stretch that follows, then move both windows forward and do it again. The result is a chain of out-of-sample segments instead of a single one. It shows whether the rules hold up as conditions change, but every segment is still past data, so it is not a forecast either.

Does a rule that held up on one market work on another?

Not automatically. Markets differ in liquidity, volatility and costs, and a rule tuned on one can depend on features the other lacks. Running the same rules, unchanged, on a different market is a useful check against overfitting: if they only work where they were built, take that as a warning. If you change the settings for each market, you are fitting again.

Is older price history still useful in a backtest?

It can be, with care. Older data adds trades and market phases that a recent window may lack. But costs, liquidity and the way a market trades can change over time, so old fee and fill assumptions may not match today. A rule that worked only in a distant stretch of history, and not in recent data, may be describing a market that no longer behaves that way.

What happens when a stop and a target fall inside the same bar?

The test has to guess. A price bar records its open, high, low and close, but not the order in which the high and the low came. If both your stop and your target sit inside one bar, the data cannot tell which was hit first. Assuming the target is the optimistic guess. A careful test assumes the stop, or checks the case on finer data.

Contents
Topics

92 articles in total

Ready to trade with our capital?

Pass the evaluation and trade funded on a prop firm built by traders, for traders.

Get Funded

More Insights

Education

What Is MEV? Who Sees Your Transaction Before It Lands

MEV is the value someone can earn by choosing the order of transactions before they land in a block. Here is where your transaction waits, how front-running, sandwiches and back-running work, what changes between pools and order books, and why a public order is not the same as a fair one.

9 min read