Your Backtest Dies Climbing Into Majors

Key Takeaways

• We ran one 5.7-year perpetual-futures search three times, changing only the coins: majors (BTC, ETH, SOL, BNB), mid-caps (XRP, DOGE, ADA, LINK), and thin small-caps (ICX, KNC, IOST, ZRX). Same 818 indicator settings, same 18 exit brackets, same candles: 144 million backtests, and every strategy has an exact twin on each tier.
• 451 rows beat their own tier's luck bar with 30 or more trades. We moved each one to the other two tiers and graded its twin three ways: against the destination's luck bar (zero passed, and we show why that zero means nothing), against a fair single-bet test (95 of 902 passed), and against the control that test demands: how often a random local row passes the same bar.
• The control is the story. Of 902 transplant checks, 712 twins had enough trades to qualify, and the destinations' own base rates say chance alone passes about 237 of those. Transplanted "survivors" passed 95. Being a statistical survivor somewhere else made a strategy worse than a random local one, on average, with one exception: shorts moving into the thinnest, hardest-falling tier passed at up to six times the local rate.
• The majors rejected everything. 379 transplant checks landed on BTC, ETH, SOL and BNB, 359 of them with enough trades to qualify, and chance alone should have passed about 103. Zero passed, from every origin, in both directions of trade.

1. The move everyone makes

You find a configuration that looks clean on BTC. The first thing you do, before position sizing, before forward testing, is point it at something smaller: DOGE, or that L1 you are holding. The theory is simple: if the logic is real, it should work even better where markets are less efficient.

Everyone does this by hand, one strategy at a time. We did it as a census. Three tiers of coins, one identical search on each, and then a lookup: take every configuration that cleared the statistical bar on one tier, and read its result on the other two. Same entry, same exit, same window. No re-optimization, no survivor stories. Every strategy, checked everywhere, against a control.

The folk theory says edges flow toward less efficient markets. The census found something harsher. On average, transplanted survivors performed worse than random local strategies on the same test. The one pocket of genuine transfer pointed at the thinnest, most distressed market on the list. And the most liquid tier rejected every single transplant, by a margin no noise can explain.

2. Three tiers, one search

The setup. Five common indicators (SMA, Ichimoku, Stochastic RSI, MACD, Bollinger Bands), swept across the 1-hour and 4-hour timeframes, give 818 configurations. Paired two at a time, that is 334,153 two-indicator strategies. Each one is tested against 18 exit brackets (take-profit 3 to 5%, stop-loss 2.5 to 3.5%, time limit 120 to 480 bars), long and short, on a four-coin basket. One run: 48.1 million backtests (6.0 million strategy rows per direction, each row backtested on the basket's four coins). We ran it three times on Binance USDT perpetuals, changing only the symbols:

• Majors: BTC, ETH, SOL, BNB
• Mids: XRP, DOGE, ADA, LINK
• Smalls: ICX, KNC, IOST, ZRX (the four thinnest perps with a full 2020 to 2026 history, around $1M of daily volume each, versus billions for BTC)

The grid, identical in all three searches: SMA period 20 to 300 (step 10); Ichimoku tenkan 7 to 12, kijun 22 to 34 (step 4), senkou B 44 to 60 (step 4); Stochastic RSI periods 7 to 21 (step 4) with smoothing 3 or 5; MACD fast 5 to 17, slow 13 to 37, signal 5 to 9; Bollinger Bands period 20 to 50 (step 2), width 1.5 to 2.5 (step 0.2). That is ten indicator-timeframe cards (five indicators on each of two timeframes) and 818 configurations, paired with the scanner's default entry rules; entries and every other engine convention follow the 100-million-backtest study.

Same 5.7-year window, same candle snapshot, same fees (0.04% taker plus 0.01% slippage on every fill), same engine. The config receipt embedded in every export is identical across the three searches except for the symbol list. That receipt is what makes the twin lookup in Section 3 legitimate.

Each of the six populations (three tiers, long and short) gets graded by its own luck bar. Search five-plus million configurations and pure chance will hand you some impressive Sharpe ratios, so a result only counts if it beats the best score luck alone would post in a search that size.¹ The deflated Sharpe ratio (DSR) that encodes this is a probability from 0 to 1: the confidence, given the row's own trades, that its true Sharpe clears that hurdle. It is a bar you cannot compute for one strategy in isolation; it needs the whole population's statistics, which is what a full scan export carries. Counting rows with 30 or more trades that cleared DSR 0.95 against their own population:

	Long	Short
Majors	4	68
Mids	12	16
Smalls	0	351

451 rows in total. We will call them survivors: rows this search could not dismiss as luck. Two honesty notes before we move them anywhere. First, 451 overstates the discovery count, because some exit variants never trigger (a 480-bar time limit behind a tight stop rarely fires), so several rows can describe one identical outcome. Folded, the 451 rows collapse to 201 distinct configurations, and we quote deduplicated counts wherever they differ. Second, if the majors column looks like it contradicts our 100-million-backtest study, where no single strategy cleared 0.95, recall the follow-up experiment: the luck bar is a property of the search you assemble, not of the market. Different grid, different window, different bar. "Survivor" means these 451 cleared this search's bar. That is all it means, and it is already more than most backtests can say.

3. Every survivor has a twin

Because the three searches share one grid, strategy #217,404 on the majors is, entry logic, parameters, exit bracket and all, strategy #217,404 on the smalls. Transplanting a survivor needs no new backtest: we look up its twin's row in the other tier's export. 451 survivors, each with twins on two other tiers: 902 lookups.

First, the naive question we set out to publish: how many survivors beat the destination tier's luck bar too? Zero. No majors survivor re-cleared on mids or smalls, no mids survivor on majors or smalls, no smalls survivor anywhere.

That zero was our working headline for about half a day, and it deserved to die. It fails twice.

Zero is what randomness predicts. Survivors are rare: 0 to 351 rows out of roughly 5.8 million gradable rows per population. Pick 451 rows at random, check them against another tier's survivor list, and the expected overlap across all twelve transplant cells is about 0.01. Observing zero carries almost no information. Any pair of unrelated searches this size would post the same zero.

And it is the wrong bar. The deflated-Sharpe threshold exists to punish search size: it asks whether a result beats the best of 5.8 million tries. But a transplanted strategy is not a search. It is one pre-specified bet. The fair test for one bet is the single-strategy version of the same statistic, the Probabilistic Sharpe Ratio (PSR, also 0 to 1), above 0.95 with the same 30-trade floor, in the destination market.²

Under that test, 95 of the 902 transplant checks pass. And that number, standing alone, is exactly the kind of statistic this blog exists to knock down. Run 902 checks at a 0.95 bar and chance alone passes dozens; how many depends on the destination. So we ran the control the number demands: in each destination, how often does a random 30-plus-trade row pass the same test?

The answer reframes everything. These markets are not stingy: 29% of qualifying rows pass on the majors short side, 43% on the mids short side, 12% on the smalls short side. (Before that reads as easy money, remember what a pass certifies: consistency, not size. A confidently-positive record under bracket exits is usually the +8.8%-in-six-years scalpel of the next section.) Weight each transplant cell by its destination's own pass rate, count only the 712 twins with enough trades to qualify, and the checks should have produced about 237 passes if survivors were merely average local rows. They produced 95.

Each transplant route: the share of eligible survivor twins (30-plus destination trades) passing a fair single-bet test (top bar) against how often a random qualifying row in the destination passes the same test (thin gray bar), with counts and enrichment at right. Only the two routes into the small-caps beat their base rate; the routes into the majors pass zero times against roughly 103 expected among eligible twins. Source: the six band exports, 2026-07-12.

4. The control table

Here is the whole experiment in one table. "Passes" is transplanted survivors clearing PSR 0.95 with 30-plus trades in the destination. A twin that traded fewer than 30 times cannot pass by definition, so "eligible" counts the twins that could; "expected" is what the destination's own base rate predicts across those twins; enrichment is observed divided by expected. Longs are pooled: only 16 of the 451 survivors are longs, too few to say much beyond "zero passed."

Transplant (shorts)	Passes	Eligible twins	Expected by chance	Enrichment
Mids → Smalls	12	16 of 16	1.9	6.2x
Majors → Smalls	14	34 of 68	4.1	3.4x
Smalls → Mids	61	251 of 351	108.1	0.56x
Majors → Mids	8	42 of 68	18.1	0.44x
Mids → Majors	0	16 of 16	4.6	0
Smalls → Majors	0	337 of 351	97.4	0
All longs, all routes	0	16 of 32	2.3	0

Deduplicated to distinct configurations, the short cells read: Majors to Mids 4 of 50, Majors to Smalls 10 of 50, Mids to Smalls 4 of 6, Smalls to Mids 25 of 139. Same pattern, smaller numbers. Three facts survive every way we count.

Genuine enrichment exists in exactly one direction: into the smalls. A mids short survivor was 6.2 times more likely than a random local row to pass on the small-caps; a majors short survivor, 3.4 times. That is real transfer, and Section 5 will show it is mostly not a compliment.

Into the mids, survivors underperformed random local rows. The table's biggest cell, 61 passes for smalls survivors moved up one rung, sounds impressive until the base rate arrives: random mids rows pass 43% of the time, eligible transplants 24%. Part of this is structural: survivors trade sparsely (their twins' median is 28 to 34 trades), and a consistency test rewards rows that trade constantly. Raw Sharpe tells the same story from another angle: transplants often sit near the top of their destination by Sharpe alone. In the most dramatic case, 58 of 68 majors-to-smalls twins land in the destination's top 1%, with a median annualized Sharpe of +4.65. Which sounds heroic until you see what it earned: about +8.8% total over 5.7 years, at a 61.5% win rate, a −2.7% max drawdown, 28 trades. The uphill cell reads the same way: smalls survivors moved up to mids earned a median +8.7% (58.8% win rate, −3.3% max drawdown, 34 trades). A +4.65 Sharpe that makes nine percent in six years is a scalpel, not a rocket, and a Sharpe estimated from 28 trades deserves suspicion on principle.²

The majors rejected everything, and not quietly. 379 checks landed on the majors, long and short, and 359 had enough trades to qualify. Their base rates say chance alone passes about 103. Zero passed. The median transplant from the mids: Sharpe −0.64, return −2.9%, win rate 35.2%, drawdown −9.5%. From the smalls: Sharpe −0.17, return −0.6%, win rate 45.0%, drawdown −7.1%. This is not "we failed to find significance." It is the strongest anti-signal in the dataset.

Where each route's transplanted twins landed: p10 to p90 whisker, interquartile band, median dot, on the destination's annualized Sharpe. Sparse traders (medians of 28 to 34 trades) make these Sharpes look large next to their modest total returns. Source: the six band exports, 2026-07-12.

Aggregates hide faces, so here are the three tier champions by deflated Sharpe, followed everywhere they went (exits read take-profit / stop-loss / time limit):

• The majors short champion: Stochastic RSI (15, 15, smoothing 5) with Bollinger Bands (46, width 1.9), both on the 1-hour, exits 5% / 3.5% / 120 bars. At home: DSR 0.988 (N = 5.82M), annualized Sharpe 15.7, +26.0% total, −0.9% worst drawdown, 31 trades at an 80.6% win rate. Its twin on the smalls stayed respectable (+6.3 Sharpe, +11.0%) but traded 26 times, four short of qualifying. On the mids it flipped sign: −1.5 Sharpe, −2.5%, win rate 40%.
• The mids short champion: Stochastic RSI (7, 11, smoothing 3) with Bollinger Bands (24, width 2.1) on the 4-hour, exits 5% / 2.5% / 240 bars. At home: DSR 0.970 (N = 5.81M), +46.3% on 85 trades. Moved down to the smalls, it is the poster child of the one enriched route: +3.7 Sharpe, +12.2%, PSR 0.958, a clean pass. Moved up to the majors, the same rules lost money: −1.1 Sharpe, −4.2%, a −11.2% drawdown, a 31% win rate.
• The census's single best row, the smalls short champion: Stochastic RSI (15, 19, smoothing 5) with Bollinger Bands (34, width 1.9) on the 4-hour, exits 4% / 3.5% / 480 bars. DSR 0.9911 (N = 5.81M), +37.4% at home with a −1.8% drawdown and a 79.6% win rate. On the mids: +6.4%, PSR 0.84, short of the bar. And on the majors, the best strategy this entire census produced earned +0.1% in 5.7 years, with a PSR of 0.52. Not a disaster. A coin flip.

None of the three is a recommendation. They are here because a table of rates cannot show what "does not transfer" feels like on a strategy you might actually have found and trusted.

5. It is mostly the market, and that is the lesson

The strongest objection writes itself: the smalls fell 85%, so of course short strategies "work" there. Correct, and we are not going to fight it. Buy-and-hold over the same window (the average of each basket's four coins): majors +1,772%, mids +812%, smalls −85.2%. Every one of the four small-caps lost between 75% and 93% of its value. Coins that fell far enough to be delisted are absent from the data entirely, so the small-cap tier you can even measure is the segment's healthier remnant.³

Per-coin buy-and-hold over the identical window, first close to last close, grouped by tier with the tier's arithmetic mean at left. The three tiers hand a short strategy three different markets: one that tripled or better, one mixed, and one that lost 75 to 93 percent on every member. Source: the shared candle window of the three scans.

Read the control table with that in hand. The one genuinely enriched route sends short strategies into a market that spent six years falling: much of that 6.2x is the destination's drift meeting the strategy's direction. And 97.8% of small-cap long backtests that traded lost money, which is why not a single long transplant found a foothold there. The below-random showing on the mids and the total rejection by the majors carve the same lesson from the other side: whatever a survivor carried out of its home market, it was not a portable edge that any destination was obliged to honor.

We cannot fully separate the causes, and it would be dishonest to pretend otherwise: our tiers differ in liquidity, volatility, listing age and drift all at once, and a fixed 3-to-5% take-profit bracket is mechanically a different test on BTC than on a coin that moves twice as much in a day. The majors' total rejection fits the picture of deeper liquidity and more heavily arbitraged prices, but on this data that is an interpretation, not a measurement. What the census does establish is bounded and useful:

A backtest score is a joint property of a strategy and a market. Not a property of the strategy alone, which is how everyone files it mentally. Move the strategy, and the part contributed by the market stays behind.

The destination decides, and the market's own numbers set the yardstick. Judged naively, 95 transplant passes looks like vindication. Judged against each destination's base rate, survivors transferred worse than random almost everywhere. If you take one habit from this article, take the control: never grade a transplanted strategy without asking what a random local strategy scores on the same test.

6. What this says about your workflow

Be precise about what each tool answered here. The deflated-Sharpe bar answered: within this search, on this basket, is the result distinguishable from search luck? It did that job in all six populations. The transplant test answered a different question: does the result persist under a regime it was not selected in? Passing the first says nothing about the second. We had 451 survivors, statistically clean at home, and abroad they underperformed random local rows everywhere except one distressed corner.

To close the loop: in-tier survival means that on this window, on this grid, we could not blame the number on search luck. It is not a certified edge, and nothing here is a suggestion to short small-caps. Most of those 351 small-cap rows stand on a market that fell 85%, with 5.7 years of perpetual funding costs unmodeled on top, and 0.01% slippage is a paper convention that a real position on a $1M-a-day book would not enjoy.³

The practical takeaway is symmetric and cheap. If your candidate came from an altcoin scan, its twin's row on the majors is the fastest regime-robustness check you can buy; in our data, that check was ruthless. If your candidate came from BTC, its twin on thinner coins tells you how much of its glow is drift and chop you could have had for free with a directional bet.

7. Run the two-tier check on your own basket

The whole experiment is a pattern you can copy with two scans and two exports:

1. Pick two baskets you actually care about: say, the four coins you hold, and four majors.
2. Run the same scan on both: same indicator cards and ranges, same exit grid, same window. Identical grids are what make row-for-row twin lookup possible.
3. Export both runs (the "Save all results" toggle). Every row carries its trade count, and on paid plans its deflated Sharpe and PSR; the file's metadata carries the population's trial count and its luck bar, and the embedded config receipt proves the two searches matched. The luck bar is the part you cannot get from testing five candidates by hand: it only exists once a whole population has been measured.
4. Take each run's survivors and read their twins' rows in the other file, graded as single bets (PSR, not the luck bar). Then run the control from Section 4: compare your twins' pass rate against how often a random qualifying row in the destination file passes the same test.
5. Before celebrating any transplant, compute both baskets' buy-and-hold over the window and ask the question Section 5 forced on us: did the strategy travel, or did the market underneath it?

Frequently asked questions

Isn't "shorts do well on falling markets" obvious? That part, yes, and the article does not lean on it. What was not obvious before the census: that zero of 451 survivors would re-clear a luck bar anywhere, and that the zero would be statistically empty; that a fair single-bet test would pass 95 transplants while the destinations' own base rates predicted about 237; that the only genuinely enriched route would point into the thinnest tier rather than out of it; and that the majors would reject all 379 attempts against about 103 expected. The folk theory predicts edges migrating to less efficient markets. The control says survivorship mostly did not migrate at all.

Why not judge transplants by the destination's deflated Sharpe? Because deflation is a penalty for search size, and a transplant is not a search. The destination's bar answers "is this the best of 5.8 million tries, or luck?" But you did not try 5.8 million times. You tried once, with a strategy chosen on different data. Hold one bet to a one-bet standard (PSR), hold searches to a search standard (DSR), and then, because even one-bet standards pass random rows at a knowable rate, compare against the local base rate. Skipping that last step is how our own first two headlines went wrong.

Should I scan each coin separately, then? Scan the basket you intend to trade, and validate on the basket you did not scan. The single most load-bearing choice in this article was keeping the grids identical across runs; that is what turns "I re-ran it and it felt similar" into a row-level join with exact twins and a computable control. The luck bar will differ per basket, which is expected: it is the composition effect we measured directly.

---

Auto-trading and trading carry a risk of losing your principal. This article is educational, does not guarantee profit, and past backtest results do not predict future returns.

Footnotes

1. The threshold is the Deflated Sharpe Ratio of Bailey & López de Prado (2014), "The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality", The Journal of Portfolio Management, 40(5). Per-population trial counts (about 5.81 to 5.82 million effective trials per direction) and the luck bar each population sets are recorded in every export's metadata. ↩

2. Same statistic family, different null: PSR grades one track record against a benchmark using its own trades, skew, and kurtosis; DSR grades the same record against the best of N searched. Both are computed per row in the export. Sharpe figures quoted in this article are annualized; per-trade Sharpe is a separate, smaller-scaled column. Sharpe-type columns are capped at the extremes (annualized at ±100, per-trade at ±6.2994), and no number in this article sits on a cap. A caution that cuts against our own survivors: with only 30-odd trades, the skew and kurtosis terms inside PSR are themselves noisy estimates, which is one more reason the base-rate control matters more than any single row's score. ↩ ↩²

3. Short legs are modeled with taker fees and slippage but without perpetual funding payments, which over 5.7 years of persistent shorting are material and usually adverse; and the flat 0.01% slippage does not model market impact or fill capacity on books this thin. Survivorship in the coin list cuts the other way: delisted coins are absent, and a delisting forces any open position closed regardless of its paper trajectory. Treat every small-cap short figure as an upper bound on paper, not an invitation. ↩ ↩²