recording-based backtests: how many days before i trust it?

read the guide on backtesting against your own recording instead of resampled bars. makes sense to me in principle, but i have questions before i rebuild my whole routine. my situation: - i trade premarket into the first 30-40 min, mostly rotation off a gap scanner - universe changes every single morning, which is exactly the thing the guide says resampled bars destroy - so far i've been backtesting on 1m bars from a separate data pull and the fills always looked nicer than reality what i'm unclear on: 1. recordings only start once you're subscribed to a symbol. for me the symbol i care about often isn't in my list until the scanner pulls it in at 6am. does the recording capture it from that moment forward, and is a partial morning still usable for a backtest, or is it garbage in? 2. how many recorded days is a sane minimum before the overfitting check on the report is even meaningful? i can imagine getting a clean equity curve off two weeks and fooling myself completely. 3. "prepare engine" - do i need to redo that every time i add new recorded days, or does it build on what's there? not asking anyone to hand me a strategy. just want to know if my morning workflow breaks the assumptions the guide is built on.
P
PremarketPete
20 posts · 0 followers
+ Follow

Related reading

4 replies

I
IndicatorSkeptic· Aug 2026 ago
so you've been backtesting on bars that never existed against fills you never got, and the results were flattering. shocking. welcome to the club, the coffee is terrible.
R
RiskFirstRita· Aug 2026 ago
on your q2 - i'd flip it around. don't ask how many days makes the p-value meaningful, ask how many *different kinds* of morning you've recorded. two weeks of the same regime is one sample wearing fourteen hats. my checklist before i believe any backtest: does it include a quiet stretch, does it include a nasty one, does it include a day my scanner gave me junk, and did i run the drawdown check *before* i looked at the equity curve. that last one matters more than people think, once you've seen the curve you're negotiating with yourself.
ZenTrader_Ana· Aug 2026 ago
the partial-morning question is the honest one and i think you already know the answer. what strikes me though is you noticed your old backtests always looked nicer than reality - that noticing is worth more than the tooling. keep a note of what you expected each morning versus what filled, separate from any report.
S
SpreadWatcher· Aug 2026 ago
partial recording is fine as long as you're honest that the backtest starts when the recording starts, not when your idea starts. the bigger thing for your case: gap rotation names in premarket have a book that looks nothing like the smooth thing 1m bars imply, and the difference between assumed and actual fill on those is a lot wider than most people budget for. that's the whole argument of the guide really.
Sign in to reply →
← All brokers & apis