Show me the trades I didn't take

When a filter removes a signal — an ML gate, a risk limit, a time window, I'd like to see what would have happened to that trade. Not to override the filter. To measure it. A filter is only worth having if it removes more bad than good, and right now I have no way to check whether mine does. I'm trusting a component I can't evaluate. Counterfactual outcomes for skipped signals would turn every filter into something testable.
DataDrivenDee
12 posts · 0 followers
+ Follow

Related reading

6 replies

MLcurious· Jul 2026 ago
This is what I'd need before I'd trust the ML gate. Which the team themselves suggested verifying rather than assuming.
Q
QuietVol· Jul 2026 ago
Careful with the interpretation though — a counterfactual on a skipped trade ignores that taking it might have changed your available capacity for the next one. Still worth having, with that caveat stated.
DataDrivenDee· Jul 2026 ago
Fair caveat, and I'd rather have the imperfect measurement with a warning than no measurement.
TraderWeTraderWe Team· Jul 2026 ago· edited Aug 2026 ago
Logging the skipped signal itself is the easy part — we already know why a signal was dropped and by which filter. The hard part is the outcome: to say what would have happened we have to simulate the exit path the same way a real position would have exited, which means fills, stops and trailing logic that never actually ran. So treat any number we show as a modelled result, not a realized one, and QuietVol's point about sequencing is real. We can't reconstruct the alternate portfolio, only the isolated trade. No date to give you, but per-filter hit/miss counts on skipped signals is the version we'd start with, since that answers your question without pretending the P&L is exact.
S
SpreadWatcher· Jul 2026 ago
The counterfactual fill is where this gets ugly. If the sim exits at mid and the real position would have crossed the spread twice plus fees, every filter you own will look like it's costing you money that never existed. I'd want the skipped-trade P&L charged the same spread and slippage assumptions as a live one, or the numbers are just decoration.
BacktestBetty· Jul 2026 ago
Useful, with guardrails: 1) needs a minimum count before the number means anything — a filter that skipped 12 signals tells you nothing either way. 2) report it as a distribution, not one aggregate figure, since one huge avoided loss can make a bad filter look great. 3) the real danger is that once you can see skipped P&L you'll start tuning the gate until the counterfactual looks bad, which is just fitting the filter to the same history twice. Maybe show it per-filter and per-period so you can at least eyeball whether it holds up out of sample.
Sign in to reply →
← All feature requests