I optimized a strategy into uselessness and didn't notice for two months

Started with something simple that made a bit of money. Added a filter, better. Added another, better. Tightened a threshold, better again. By the end it had eleven conditions and took four trades a month, all of which had been carefully shaped to match the four best trades in my test period. Live, it took two trades in two months and lost on both. The number that would have warned me: trade count kept falling while returns kept rising. I was watching the second one.
D
DrawdownDave
11 posts · 0 followers
+ Follow

Related reading

4 replies

DataDrivenDee· Jul 2026 ago
Falling trade count with rising returns is close to a definition of overfitting. Good catch, painful way to learn it.
H
HalfKelly· Jul 2026 ago
Worth tracking returns per trade rather than total. It makes this failure visible immediately.
Q
QuietVol· Jul 2026 ago· edited Aug 2026 ago
Eleven conditions fitted to what ends up being four observations, the effective degrees of freedom are basically zero, so the backtest curve is mostly memory, not estimate. I try to set a floor on trade count before I look at returns at all, otherwise the confidence interval is wider than the effect. Did the equity curve get smoother as you added rules, or just steeper?
MLcurious· Jul 2026 ago
Oh this is exactly the thing my course kept calling out and I only half understood until now. Dumb question but how do you decide when a filter is a real edge vs just fitting noise? I've been doing the train/test split thing but I keep peeking at the test set when something doesn't work, which I'm guessing defeats the whole point lol.
Sign in to reply →
← All strategies & backtesting