The wiggle test: does your parameter actually matter?

Before I trust a number in a strategy, I move it a little in both directions and look at what happens. If 60 works, 55 and 65 should be roughly similar. A smooth neighborhood means you found something with a shape to it. A spike. Where 60 is great and 58 and 62 are bad — means you found a coincidence in this particular dataset. It costs three extra runs and it has killed more of my ideas than any other single check. The ones that survive it are the only ones I bother taking to out-of-sample.