dead minutes — do i drop them or flag them?

so i read the research post here about how much of a recorded session is basically empty minutes (nothing traded, quote never moved) and i've been staring at my feature pipeline ever since feeling kind of dumb. my setup: small fixed watchlist, 1 minute bars, i build rolling features (returns, volume z-score, spread) and train a little gradient boosted thing to predict the next few minutes. the post's argument, as i understood it, is that a big share of those minute rows on a fixed list aren't observations at all, they're the previous observation with a new timestamp on it. which if true means my rolling windows are being padded with copies of themselves, and my train/test splits look way bigger than they really are. what i don't know is what to actually DO about it: 1. drop those rows entirely and let the clock be event based instead of wall clock? feels cleaner but then my "last 20 bars" is sometimes 20 minutes and sometimes most of an afternoon, and i have no idea if the model can handle that without me feeding it elapsed time as a feature. 2. keep them but add a flag column saying "nothing happened here"? that keeps the shape but i suspect the model just learns the flag and i've solved nothing. 3. resample to something like volume bars / dollar bars, which people keep mentioning to me and i keep not doing because i don't fully understand them. also the part that stuck with me was that a scanner-driven universe had way fewer of these empty minutes than a fixed list. is that just because scanners pick things that are already moving, i.e. it's selection and you've now got a survivorship problem baked in? because that feels like trading one bias for another, not fixing anything. and honestly the bit i'm least sure about: does any of this even matter for a slow strategy? if i'm holding for hours, do i care that the minutes in between were dead? or is the damage all in the evaluation stage, where a backtest counts a stale minute as a real chance to be right? happy to be told i'm overthinking a beginner problem.
MLcurious
18 posts · 0 followers
+ Follow

Related reading

3 replies

TraderWeTraderWe Team· Aug 2026 ago
Not overthinking, it's a real thing and it bites people at the evaluation stage more than the entry stage. Two notes from our side: our historical bars are carried forward when there's no print, so an unchanged bar in the file genuinely can mean "no activity" rather than "activity that netted to zero" — you can usually separate the two by checking whether traded value moved at all, which is roughly the test the research post used. What we don't expose cleanly right now is a per-bar flag telling you which case you're in, so you'd have to derive it yourself. On your options: the flag column approach is the least destructive if you're still exploring, because you can always drop later, but you can't un-drop. And yes, evaluating a rule against a bar where nothing could have happened is the part that quietly inflates results.
DataDrivenDee· Aug 2026 ago
The selection question is the one I'd want to pin down first, because it's testable and the rest is kind of downstream of it. Something like: take your scanner universe, and instead of judging it on how few dead minutes it has, judge the same strategy on the fixed list with dead minutes dropped versus the scanner list untouched. If the scanner is genuinely giving you more information rather than just more motion, the edge should survive the comparison. If it only looks good on the scanner side, you've probably found the survivorship thing you're worried about rather than a data quality fix. On your slow strategy question I'd guess it matters less for holding and more for your labels, since a label computed over a window that was mostly stale isn't measuring what you think it is.
N
NightOwl_Yuki· Aug 2026 ago
crypto side i basically never see this, it's 24/7 and something is always printing, so the whole dead minute concept was new to me when i read that piece. that said the overnight hours here get thin enough that my indicators go weird in a similar way, so maybe it's the same illness with a different rash. fwiw i'd go with the flag column first just so you can see how much of your data it actually eats before you commit to throwing rows away.
Sign in to reply →
← All q&a