Hitting the nail with your head

I smiled when reading a recent comment. In reference to my suggestion that cables make a sonic difference the commenter blurted, "when done properly people can't hear the difference." Bingo. When we devise a test procedure like the double-blind switched A/B test that nulls the differences in devices under test, it's the procedure we should be questioning instead of the results. Think of it this way. Let's imagine we have a consistent observation we'd like to test. Perhaps it's that we somehow act differently during a full moon than a new moon. That the moon has an effect on us whether we can see it or not. We identify at random 100 people within a community who agree to be part of our test. Each month our test subjects take notes of any changes in behavior on the day of the full moon and compare them to those on the day of the new moon. At the end of a year, we compare notes and find there's statistically no difference. Thus, we conclude there are no differences. We are unaffected by the cycles of the moon. And yet, we know this to be false. Human sleep patterns are disrupted during the cycles of the moon—even when the moonlight is not visible. The problem with our test has more to do with the methodology than proving the moon hasn't any effect. Had we been more specific and recorded sleep patterns we would have found very different results. So now imagine we start with the observation that for a large segment of the audiophile community cables sound different. We must first identify under what conditions they sound different and then see if we can duplicate those in a way that removes opinion and conjecture. For example, it's likely we need trained listeners. Next, we need resolving systems. Then, subjects must be able to spend what time they need to observe differences, selecting between two unknown variables at their leisure. Lastly, we need to choose the music and environment. My point is simple. If we set out to prove there are no differences we devise one sort of test. If we set out to prove there are differences, we have yet another. A good scientist seeks the truth.