so how long do i have to record before it's actually usable?
ok that guide on recording your own crypto data has been rattling around my head all week!!
The part that got me was the bit about volume being reported wildly differently between venues for the SAME symbol. Not a little different. Enough that any threshold I tuned on one source would be nonsense on the other. I have absolutely been guilty of pulling free history from wherever and pretending it's the market I actually trade on. Guilty! Moving on!
So I set up a recorder on a cheap little box last weekend and it's been ticking away. But here's where I'm stuck, and the guide sort of stops before answering it for me:
1. How much do I actually need before I stop treating it as a toy? A week of my own feed is my own feed, sure, but it's also one week of one regime. Momentum stuff is all about how it behaves when things get loud, and quiet weeks tell me nothing about loud days.
2. Gaps. My box dropped connection twice already (my fault, not the venue). Do I patch those, mark them and skip them, or throw the whole session? Feels like silently patching is how you end up backtesting a fantasy.
3. The depth argument — I get why order book history is the interesting bit and why nobody sells it. But I mostly hold for minutes, not seconds. Is depth recording worth the storage for someone on my timeframe or am I hoarding data I'll never open?
Anyone actually done this for a few months and looked back at their early files? What did you wish you'd recorded from day one?!