Backtesting a Strategy Honestly

Backtesting means running a trading strategy against historical price data to see how it would have performed. Done honestly, it is the cheapest way to kill a bad idea before it costs you money. Done carelessly — which is the default — it produces beautiful equity curves that evaporate the moment real money touches them, because the test quietly assumed knowledge, fills, and costs that no live trader ever gets.

Why Most Backtests Flatter You

A backtest is a simulation, and every simplification in that simulation tends to err in your favor. Historical candles don't show the spread you would have crossed. The strategy you're testing is usually the survivor of dozens of discarded variants, so its results are already selected for luck. And the coins you chose to test are typically the ones that did well enough to still be on your radar.

None of this requires dishonesty. It happens through a series of individually reasonable choices: "I'll just tune this threshold," "I'll test on BTC since it has the most data," "I'll ignore fees for now and add them later." Each choice injects a small optimistic bias, and the biases compound. A strategy that shows +80% over two years in a naive backtest can easily be flat or negative once you account for the four problems below. The goal of honest backtesting is not to eliminate every bias — that's impossible — but to know which direction each one pushes and to keep them all pushing against you rather than for you.

Overfitting: Tuning Until the Past Looks Perfect

Overfitting is fitting your strategy to the noise in one particular slice of history rather than to anything repeatable. It is the single most common way backtests lie.

The mechanism is simple. Suppose you test an RSI mean-reversion bot and try buy thresholds of 20, 25, 30, and 35, with take-profits of 2%, 3%, and 5%, on three timeframes. That's 36 combinations. Even if the underlying idea has zero edge, one of those 36 will look great on any given dataset purely by chance — and that's the one you'll be tempted to keep. You didn't discover an edge; you ran a lottery and kept the winning ticket.

Warning signs of an overfit strategy:

Suspiciously specific parameters. RSI 27.5 with a 3.8% take-profit performs wonderfully but RSI 25 or 30 loses money. A real edge degrades gradually as you move parameters; a fake one falls off a cliff.
Many rules, few trades. A strategy with six entry conditions that fired 40 times in two years has roughly one rule per seven trades. That's memorization, not a strategy.
Performance concentrated in a handful of trades. If removing the three best trades turns the equity curve flat, you backtested a few lucky events.

The defenses are equally simple, if uncomfortable: prefer fewer parameters, test whether neighboring parameter values also work (a plateau of decent results is worth more than one spectacular spike), and demand a sample of at least a few hundred trades before trusting any statistic.

Look-Ahead Bias and Its Relatives

Look-ahead bias means your simulation uses information that wasn't available at the moment of the simulated trade. It's often invisible in code and always fatal to validity.

Common forms:

Trading on an unclosed candle. An indicator computed on a candle's close isn't known until that candle closes — often the single most common bug in home-built backtests. If your test buys at that same candle's close price the instant the signal forms, fine; if it buys at that candle's open or low, you've time-traveled.
Repainting indicators. Some indicators (certain zig-zag, fractal, or smoothed signals) recalculate past values as new data arrives. Backtested on final historical values, they look prophetic; live, the signal you acted on redraws itself.
Survivorship bias. Testing only on today's top-20 coins means testing only on assets that, by definition, didn't collapse. The 2021-era portfolio that included coins which later fell 95% or were delisted looked very different from what a top-20 screen shows today.
Data snooping across attempts. Every time you peek at full-history results and then adjust the strategy, the full history stops being a fair test. You've turned all your data into training data.

The structural fix is out-of-sample testing: lock away a portion of history — say, the most recent 30% — before you start tuning. Tune only on the older portion, then run the finished strategy once on the held-out data. If performance collapses out-of-sample, the strategy was memorizing, not generalizing. Walk-forward testing extends this: repeatedly tune on a window, test on the next slice, and roll forward, so every tested period was genuinely unseen at tuning time.

Fees, Slippage, and the Cost of Trading Often

Cost modeling is where honest and dishonest backtests diverge most in raw numbers, because costs scale with trade frequency and small per-trade drags compound viciously.

Work an example. A bot averages three round-trip trades a day. Taker fees of 0.10% per side cost 0.20% per round trip. Add a realistic 0.05% per side for spread and slippage on a liquid pair — more on thin ones — and each round trip costs roughly 0.30%. At three trades a day, that's about 0.9% of traded capital per day in friction. A strategy whose average winning round trip nets 0.5% before costs isn't marginal — it's a donation machine, regardless of how smooth its zero-fee equity curve looks.

An honest backtest therefore:

Charges the taker fee unless the strategy genuinely rests limit orders, and models the real risk that resting orders don't fill when price runs away.
Adds slippage that scales with order size and market conditions — fills during a volatile breakout are worse than the candle print suggests, and stop-market orders in a crash can fill far below the stop level.
Uses fee tiers you'd actually have, not the VIP tier from an exchange's marketing page.
Re-runs the test with costs increased by 50% as a stress check. A real edge survives; a cost-sensitive mirage doesn't.

Judging the Results Like a Skeptic

Even a clean backtest needs skeptical reading. Total return is the least informative number on the report.

Look instead at maximum drawdown (would you truly have kept the bot running through a 40% dip in year one?), the number of trades (fewer than ~100 means wide error bars on everything), and how returns are distributed across time and assets. Compare against a dumb benchmark: if simply holding the same asset over the same period returned more with less complexity, the strategy added negative value. Check regime dependence by splitting results into rising, falling, and sideways periods — many crypto strategies are just "long during a bull market" wearing an indicator costume.

Finally, treat backtest performance as a ceiling, not an estimate. Live results are almost always worse: markets shift, execution disappoints, and the residual optimism you couldn't purge comes due. A common sanity rule is to require a strategy to look clearly good — not marginal — after all costs and out-of-sample testing, then start live with a small size, compare live fills against simulated ones for a few weeks (paper trading first helps), and scale only if reality tracks the simulation.

Key Takeaways

Backtests default to lying in your favor; your job is to find and reverse each optimistic bias, not to admire the equity curve.
Overfitting is the top killer: prefer few parameters, demand hundreds of trades, and trust parameter plateaus over spikes.
Look-ahead bias — unclosed candles, repainting indicators, survivorship-filtered coin lists — invalidates a test completely; hold out recent data you never tune on.
Model taker fees, spread, and slippage per round trip; high-frequency strategies often die from a realistic ~0.2-0.4% cost per trade.
Treat backtest results as a ceiling: benchmark against buy-and-hold, stress-test costs, and go live small before believing anything.