huice.org

Backtests Look Plump, Live Trading Looks Lean: Winning in the Past Doesn't Mean Winning Tomorrow

2026-07-20 18:53:41

Backtesting is a good thing. It lets you run an idea through historical data before you risk real money — tools like huice.org, for instance, take your strategy and tell you what it would have earned over some window, and how deep the worst drawdown got. But the sneakiest part of backtesting is how easily it "looks gorgeous."

The first trap is overfitting. You keep tweaking parameters until the curve looks absurdly good across the last ten years — but what you've fit is coincidence, not a law. It's like memorizing the answer key before an exam; change the questions and you're lost. The second trap is survivorship bias. The ETF you backtest is alive and kicking today, but plenty of its peers from a decade ago got liquidated — you only tested the "survivors," the honor students, so the result skews rosy. The third trap is look-ahead bias, plain talk for accidentally using data that wasn't visible at the time, like peeking at the answer sheet during an open-book test. Winning that way earns no respect.

So here's the right way to use a backtest: treat it as a funnel that filters out obviously broken strategies, not a certificate proving you'll get rich. A strategy that can't even beat "do nothing and just hold" in history has no real shot live; but a strategy that looks heroic in history at most shows "this trick worked back then" — nobody signs for the future. When you do step in, test with small money first and let reality teach you a lesson. That lesson costs, but it beats misty-eyed self-congratulation over a pretty backtest curve.