Survivorship Bias: The Silent Killer of Backtest Results

Backtests often lie in a specific way. They erase the losers that disappeared. Elton, Gruber, and Blake documented this clearly in mutual fund data, showing that survivorship bias materially overstated average returns when only surviving funds remained in the sample.^[1]

The same problem appears in stock research whenever someone applies today's ticker list to the past. That universe is already filtered by survival. Real investors never had that luxury.

How the Bias Enters a Backtest

Survivorship bias occurs when a dataset keeps only the entities still present at the end of the observation window. In equities, that means bankruptcies, delistings, distressed mergers, and prolonged underperformers vanish from the record. The distortion is usually upward because the names that disappear are disproportionately weak.

Carhart extended the analysis and found that the problem can be even larger in some investment styles.^[2] The same principle applies to stock universes built from current index constituents or free datasets that lack delisted securities.

Why It Matters for Equity Research

Suppose someone runs a backtest over the S&P 500 using the companies in the index today. Every firm that collapsed, merged away, or fell out of the index is already gone from the study. That means the simulation never has to own the names that hurt real investors most.

That is why academic datasets such as CRSP explicitly preserve delisted securities and their terminal returns.^[3] Once those observations are dropped, the conclusions become less credible than they first appear.

It Rarely Travels Alone

Survivorship bias often appears alongside look-ahead bias and selection bias. Look-ahead bias uses information that was unavailable on the rebalance date. Selection bias promotes only the strategies that happened to look good after many failed attempts were discarded.

Banz and Breen showed how database construction choices can manufacture apparent anomalies.^[4] In practice, rigorous research demands controls for all three problems at once.

How to Spot the Problem Quickly

Ask three questions. What dataset was used? How was the universe constructed on each date? Are delisted returns included? If those answers are vague, optimistic results deserve skepticism no matter how elegant the strategy sounds.

Why This Should Change Your Standards

Backtesting is still useful. It just stops being useful when the dataset is quietly flattering the strategy. If your research starts from current ticker lists or free histories with unclear delisting treatment, the results are probably overstated. For a related discussion on how performance itself can be summarized poorly, see Sharpe Ratio vs. Calmar Ratio.