Strategy with a typo'd factor name runs instead of failing

Steps: Write a condition referencing a factor name that doesn't exist (I misspelled one), run a backtest. Expected: An error telling me the name is unknown. Actual: It ran. Produced results. The condition involving the bad name apparently just never became true, so I got a valid-looking backtest of a strategy that wasn't the one I wrote. Impact: I spent two days concluding an idea didn't work. The idea was never tested.
MLcurious
18 posts · 0 followers
+ Follow

Related reading

10 replies

TraderWeTraderWe Team· May 2026 ago· edited Aug 2026 ago
This is the most important report on this board and we're treating it that way. A silent no-op on an unknown name is worse than a crash. It produces a result that looks trustworthy. Unknown factor and function names will fail loudly before the run starts, and the editor will flag them as you type. Sorry about the two days.
BacktestBetty· May 2026 ago
Silent-wrong is the worst failure class in research tooling. Very glad this is being fixed.
G
GapHunterMike· May 2026 ago
this one probably cost people more than any crash ever did.
TraderWeTraderWe Team· Jul 2026 ago
Update: this shipped. Unknown factor and function names now fail in two places — the editor marks them with an ✗ as you type, and a run that reaches an unknown name stops with an "undefined name" error instead of quietly treating the condition as false. The silent no-op that cost you two days is gone.
F
FiveMinFiona· Aug 2026 ago
Good catch, and worth re-running anything you shelved in the last few months. I keep a note next to each backtest with the date I ran it — makes it easy to go back and check which ones were done under a broken assumption.
C
CoffeeAndCharts· Aug 2026 ago
MLcurious thanks for writing it up properly with the steps, that's why it got fixed fast. And don't feel bad about the two days, half of us would have just shrugged and moved on to the next idea without ever finding out why.
P
PremarketPete· Aug 2026 ago
- re-test the shelved ones - keep the failed version saved too, so you can diff - ✗ in the editor is the useful half of this for me
H
HalfKelly· Aug 2026 ago· edited Aug 2026 ago
The subtle part is that a condition that's always false doesn't just do nothing. It changes what the rest of the logic means. If your entry was A and B and B silently vanishes, you tested A alone, which is a different strategy with a different trade count and completely different sizing implications. Doesn't even look wrong on the equity curve.
S
SpreadWatcher· Aug 2026 ago
Always-false conditions also quietly reduce your trade count, which makes the remaining stats look cleaner than they are. Fewer trades, less friction, prettier curve. Nothing about it screams broken.
L
LurkerLee· Aug 2026 ago
does the ✗ catch a name that exists but is spelled right for the wrong thing, like a factor that only applies to crypto used on a stock strategy? or is it purely unknown-name checking?
Sign in to reply →
← All bug reports