How do you decide a research result is real?

Minimum procedure I'd accept: state the rule before looking, count how many alternatives you considered, test on a period you haven't seen, and check the result survives removing the best few instances.

Add: predict what you should NOT see if you're right. A pattern that explains every outcome explains nothing.

That last one is the piece I was missing. Thank you.

write it down before you test. date it. if the rule needs a paragraph to explain it's already curve fit. also count your failed ideas, mine is like 20 to 1 and that's the honest number.

Adding one to QuietVol's list: decide the size you'd trade it at before you know the result, and keep it small enough that being wrong is boring. Most "real" patterns I found died at the sizing stage anyway, because the edge was too thin to survive costs. And write down in advance what would make you stop trading it — otherwise you'll just keep redefining the pattern as it fails.