Guide · Backtesting · Updated 2026-07-05

The backtesting mistakes that make retail strategies look profitable when they are not

This is a companion to the landing page on backtesting trading strategies in India, which frames why disciplined testing matters. Here the job is narrower and more forensic: to name each error that quietly inflates a backtest, explain the machinery that makes it happen, and give the fix. The through-line is simple. A backtest is a measurement, and like any measurement it can be biased. The value of the exercise is not that a good number promises profit; it is that a rigorous test can rule a fragile idea out before it costs you. Getting there means removing the specific leaks below, in roughly the order they do the most damage.

Mistake 1: lookahead and future-leak bias

Lookahead bias is the use, at the moment of a decision, of information that was not actually available at that moment. It is the most common and the most flattering error in retail code because it is invisible in the output: the equity curve is smooth, the statistics are strong, and nothing looks wrong. The strategy is simply trading on the future.

The textbook form is the same-bar leak. Suppose a signal is a moving-average crossover. You can only know that a bar's close produced a crossover after that bar has closed, which means the earliest you can act is the open of the next bar. Naive code multiplies the signal on a bar by the return of the same bar, so it buys at a price it could only have observed in hindsight. The fix is a single lag, shifting the position by one bar so that today's decision earns tomorrow's return:

signal   = (ema_fast > ema_slow).astype(int)   # known at bar close
position = signal.shift(1)                       # act on the NEXT bar
ret      = close.pct_change()
strategy = position * ret                        # no future leak

The same-bar leak is only the obvious case. Three quieter forms of future leak are just as damaging and much harder to spot:

• Restated fundamentals. A company's reported earnings or book value can be revised after the fact. If your backtest reads the final, restated figure as though it were known on the original announcement date, you are trading on numbers that did not exist yet. Point-in-time fundamental data records what was actually known on each date.
• Index-membership hindsight. Filtering a historical universe by today's index constituents leaks the future: you are only holding names that were later selected into the index, which is a form of survivorship discussed next.
• Full-series indicators and scaling. Computing a normalisation, a z-score, or a maximum over the entire dataset, then applying it to early dates, quietly feeds later data backwards. Any statistic used at time T must be computed only from data up to T.

The discipline that prevents all of these is the same: every value a decision touches must be one that a clock stopped at the decision time could have produced. That upstream care about what was knowable when is exactly what the method we teach is built around, because a rule tested on the future is not a rule at all.

Mistake 2: survivorship bias

Survivorship bias is testing on the instruments that still exist rather than the ones that existed during the test window. Names that were delisted, wound up, merged away, or dropped from an index leave the dataset, and what remains is by construction the set that did not fail. The sample is quietly tilted toward good outcomes, and the adverse cases that a live trader would have held are simply absent.

The mechanism is worth stating plainly, because it is not only about bankruptcies. Index membership churns every year: names are added and removed as size and liquidity change. A backtest run on the current members of a broad index, applied backwards, holds a basket that was selected precisely because those names did well enough to still be in the index today. The weak members that were demoted along the way never appear, so the strategy collects the benefit of their survival without ever paying for their decline.

Mistake 3: the corporate-action phantom crash

This is the error that produces the most convincing fake, and the one most specific to how Indian price data is often stored. On the ex-date of a bonus issue or a stock split, the quoted price steps down sharply, not because the company lost value, but because the same value is now spread across more shares. An unadjusted series records that step as an overnight fall, and a backtest reading raw prices treats a bookkeeping event as a market crash.

The arithmetic is exact. In a one-for-one bonus, every holder receives one new share for each share held, so the share count doubles. Total value is unchanged, so the price per share must halve. A holding of 100 shares near a price of 500 is worth 50,000 before the ex-date; after it, the holder owns 200 shares near a price of 250, still worth 50,000. Nothing happened to the position. But an unadjusted series shows the price going from about 500 to about 250 overnight, a drop of roughly fifty percent, and a naive backtest can either fire every stop-loss in the book or record a phantom loss on a move that no holder actually experienced.

The same logic applies to splits, rights issues, dividends and consolidations, each with its own factor. A total-return series folds dividends back in; a price-adjusted series applies the split and bonus factors. The one thing you cannot do is run a strategy over raw quoted prices and trust the result across any name that had a corporate action in the window, which over a multi-year Indian equity backtest is a large share of the universe.

Mistake 4: overfitting and multiple testing

Overfitting is fitting the noise of one sample instead of a pattern that repeats. In practice it arrives through multiple testing: you sweep a grid of parameters, or try many rules, and keep the best in-sample performer. The problem is statistical, not moral. The maximum of many noisy results is biased upward even when none of the candidates has any real edge, because you have selected on luck as well as on signal. Report only the winner and you report a number that will not repeat.

The tell is the shape of the result surface. A rule with a genuine edge is robust to small changes in its settings, so its performance forms a broad plateau: neighbouring parameter values give similar results. An overfit rule shows a single lucky peak surrounded by cliffs, where nudging a parameter one step collapses the result. Sweeping the grid and looking at the whole surface, rather than its maximum, is the cheapest overfitting check there is.

Mistake 5: data-snooping and the deflated Sharpe ratio

Multiple testing has a precise statistical consequence that a raw performance metric hides. If you evaluate many strategy variants and keep the best Sharpe ratio, that best value is upward-biased purely by selection, even if every variant is noise. This is the data-snooping problem, and it means a headline Sharpe cannot be read at face value without knowing how many trials produced it.

The deflated Sharpe ratio, introduced by David Bailey and Marcos Lopez de Prado in 2014, is the formal correction. Its logic, framed conceptually here rather than by formula, is that you should first ask what the best Sharpe would be if none of the candidates had any skill at all. Under their False Strategy Theorem, the expected maximum Sharpe from a set of purely random strategies rises as the number of trials rises: test enough noise and some variant will look excellent by chance. The deflated Sharpe compares your observed best against that inflated benchmark, and it also adjusts for two things a plain Sharpe ignores: non-normal returns (negative skew and fat tails make a given Sharpe less trustworthy) and the length of the track record (a short sample carries more luck). The result is a probability that the strategy's skill is real rather than selected.

The practical takeaway does not require the mathematics. A Sharpe that looks strong across a hundred parameter combinations can be statistically indistinguishable from luck once the count of trials is accounted for. So the number of variants you tried is itself a material fact about the result, and a metric reported without it is incomplete. Related tools in the same family, such as the probability of backtest overfitting, estimate directly how often the in-sample best turns into an out-of-sample laggard.

Mistake 6: transaction costs, slippage and market impact

A backtest that assumes free trading and perfect fills overstates every strategy, and the overstatement grows with turnover. Real execution in Indian markets carries a stack of frictions that must be charged on every round trip. The regulator's own data underlines the stakes: in its July 2025 study of the equity derivatives segment, SEBI found that about 91 percent of individual traders lost money in FY25, with aggregate net losses of roughly 1,05,603 crore rupees after costs, up from 74,812 crore the year before. Costs are not a rounding error; for active traders they are frequently the difference between a positive and a negative result.

The components below are the recurring pieces of the Indian cost stack. The exact figures depend on the segment, the product and the tariff in force, so the table lists what each charge is and how it behaves rather than a single rate to memorise. For the full breakdown, see the companion on the real cost of an Indian trade.

The minimal correct pattern charges a cost every time the position changes, so that turnover is penalised where it happens:

trade    = position.diff().abs()        # 1 whenever the position turns over
cost     = trade * round_trip_cost      # per-turnover friction, as a fraction
net      = strategy - cost              # what the account would actually keep

The point is not the exact number plugged into round_trip_cost; it is that a strategy whose edge is smaller than its trading friction is not an edge at all. The table below shows the shape of what costs do to a high-turnover rule. The figures are illustrative, chosen only to show the arithmetic, and are not a claim about any real strategy.

Notice how the same rule can flip from attractive to unviable purely because the cost line was added. In the derivatives segment in particular, where the friction stack is heaviest, a rule that trades often needs a large gross edge simply to break even after costs. Any backtest that skips this step is measuring a market that does not charge for participation, which is not the market anyone trades in.

Mistake 7: in-sample, out-of-sample and walk-forward discipline

The final error is procedural, and it is the one that ties the others together. If you design, tune and judge a strategy on the same stretch of data, the result is optimistic by construction, because you have already fitted to whatever that stretch happened to contain. The correction is to separate the data you learn from from the data you judge on.

In-sample data is the period a strategy is developed and optimised on. Out-of-sample data is a later, untouched period used once, as a genuine test of whether the rule survives outside the conditions it was fitted to. The trap is that out-of-sample data stops being out of sample the moment you use it to tweak the rule and test again; do that a few times and you have simply overfitted to the second period as well.

Walk-forward testing formalises the separation and repeats it. You optimise on a window, test on the next unseen window, then roll both windows forward and repeat across the whole history. Stitching the out-of-sample segments together yields an equity curve that reflects how the rule would have adapted through time, which is far closer to live behaviour than any single fit. Walk-forward comes in two shapes: an anchored window that starts at the beginning and grows, and a rolling window of fixed length that moves forward and forgets old data.

Even a clean walk-forward is not a promise. It cannot reproduce order rejections, partial fills, latency or the way liquidity shifts under stress, and markets change in ways no historical window contains. This is the honest limit of the whole exercise. A rigorous backtest earns its keep by ruling fragile ideas out, not by certifying that a survivor will earn anything, and that distinction is the difference between using a backtest and being used by one.

The catalogue, in one place

The seven errors above are easier to audit as a single checklist. Each row names the mistake, the mechanism that makes it inflate a result, what it inflates, and the fix.

Passing all seven is the minimum before a backtest deserves any weight. Any one missing is a documented cause of the retail gap between a backtest that looked profitable and a live account that was not. For the mechanics of building a test harness that avoids these in code, the companions on backtesting Indian equities with pandas and building a first trading system go step by step, and the overview of how to start algo trading in India sets the wider path.

Where this fits

Backtesting sits at the boundary between an idea and a decision to risk capital on it. Everything in this article is really one instruction stated seven ways: the test must not know anything the trader could not have known, and it must charge for everything the trader would actually have paid. A backtest that honours both is a filter that removes fragile ideas cheaply. A backtest that honours neither is a machine for manufacturing confidence in results that do not exist.

The skills that make the difference are upstream of any single test: knowing which data is point-in-time, understanding why a stop can fire on a phantom move, counting the variants you tried, and reading an equity curve for the boundary where fitting ends. Those are the habits a serious curriculum builds, and they matter more than any one clever rule, because they are what keep a rule honest.

Sources

• Deflated Sharpe ratio and the False Strategy Theorem. David H. Bailey and Marcos Lopez de Prado, The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality (2014), establishing that the expected maximum Sharpe from many random trials rises with the number of trials, and that a raw Sharpe must be deflated for trial count, non-normality and sample length. papers.ssrn.com
• SEBI equity-derivatives loss study. SEBI study of individual trader profit and loss in the equity derivatives segment, July 2025: about 91 percent of individual traders lost money in FY25, with aggregate net losses of roughly 1,05,603 crore rupees after transaction costs, up from 74,812 crore in FY24. Establishes the scale of real-world trading friction. sebi.gov.in
• Corporate-action price adjustment. On the ex-date of a one-for-one bonus issue the share count doubles and the quoted price roughly halves while a holding's value is unchanged, which produces a phantom fall on any unadjusted series. This is the standard mechanics of a bonus issue as described by Indian exchanges and brokers. Bharath Shiksha: what is a bonus issue
• Walk-forward validation. The optimise-test-roll procedure, in anchored and rolling forms, and the reservation of out-of-sample data as the only honest test, follow the standard treatment of walk-forward analysis in the trading-systems literature.

Frequently asked questions

Related reading

• Backtesting trading strategies in India: the companion landing page
• Backtesting Indian equities with pandas, step by step
• Building your first trading system
• The real cost of an Indian trade