Learn · lessons from the data

What 11 crypto backtests show: six lessons for traders

We tested eleven popular crypto trading ideas with the rules written down before the data, trading costs, a control for luck and a second period. None showed an edge. The interesting part is why they failed - the same few reasons came back again and again. They apply to any strategy you test yourself.

In short: a high win rate is not a profit; hindsight makes charts look predictive; small samples always contain a "special" month; edges fade on new data; a few tail trades decide the result; and a signal must beat a simpler version of itself. All results: 11 strategies, tested.

1. A high win rate is not a profit

Four of our tests won more than half of their trades - and none of them made money we could tell apart from chance after costs. The usual win was small; the occasional loss was huge.

The positive averages are not significant: shorting pumps averaged +1.1% with t = 1.03, shorting new listings +0.61% with t = 0.24 (t needs to be above about 2). Shorting pumps lost more than the whole position on 27 of 1,215 trades; the three worst new-listing shorts lost 219-290%. What to check instead: the average result after costs, its t-statistic, and the worst five trades.

2. Hindsight makes a chart look predictive

The bitcoin rainbow chart is drawn on the whole price history. Buy in the blue bands and sell in the red ones with that finished chart and $1 becomes $397 from 2016 to 2026. Redraw the bands using only prices known at each date and the same rule turns $1 into $22 - while simply holding made $193. The rainbow test.

What to check: could every input of the rule have been known at the moment of the trade? Indicators fitted on the full history, "tops" and "bottoms" marked after the fact and levels drawn later all fail this.

3. Small samples always contain something special

Bitcoin closed October higher in 10 of 13 years - but shuffle the month labels and an October that good appears 12.5% of the time (Uptober test). Each altcoin's best calendar month is matched by shuffled data in 20-71% of cases (best month test). Altcoin season was followed by altcoins beating bitcoin only after 3 of 8 readings (altseason test). The Bull Market Support Band averaged +4.7% over 90 days after 17 touches, but the median was +0.4% and one episode carries the average (band test).

What to check: how often does chance produce a result this good on the same data? With twelve months, one is always the best.

4. Edges fade on new data

Buying a leverage flush - open interest down 10% and price down 5% in four hours - averaged +1.32% a day from January 2024 to June 2025 across 2,452 cases, with every statistical check passed. From July 2025 the same rule averaged -0.47% over 3,441 cases. Tested only on the first period, it would have looked like a discovery. The flush test.

What to check: split the history before you look at it, build the rule on the first part, and judge it only on the second.

5. A few tail trades decide the result

In the flush test the median trade lost 0.98% and only 45% of trades won; a handful of big rebounds on market-wide crash days produced the positive average. Averaging the 905 trading days first instead of the 5,893 trades turned it into -0.08%. In the pump test the opposite tail - shorts that kept rising - wiped out the many small wins.

What to check: the median as well as the mean, the worst trades, and whether the result survives when trades on the same day are counted once. With leverage, one tail trade ends the account - see how often each leverage was liquidated in the liquidation calculator.

6. A signal must beat a simpler version of itself

Adding "open interest up 10% in a day" to extreme funding changed nothing: shorts averaged -0.34% against -0.37% for funding alone (test). Adding an open interest collapse to a 5% price drop changed nothing either: +0.27% against +0.22% for any 5% drop. A second condition can make a story sound better without adding information.

What to check: compare the full rule with the same rule minus each condition. If removing a condition does not hurt, it was decoration.

A checklist before you trust a backtest

1. Write the rule down first - entry, exit, holding time and what counts as a pass - before looking at results.
2. Use only data known at the time of each trade.
3. Subtract costs: fees, slippage and funding for perpetual futures.
4. Compare with chance: random entries, shuffled labels or the opposite side on the same data.
5. Hold out a second period and judge the rule only there.
6. Look at the median and the worst trades, not only the average and the win rate.
7. Remove each condition and see whether the result gets worse.

This is the method behind every test on this site: methodology. Test your own guesses first in the Edge or Not? quiz.

FAQ