Testing of the null hypothesis (denoted H0) is a fundamental but rarely discussed aspect of the scientific method. In its most simple form, it can be stated that you cannot prove something; only disprove it. A common corollary of this idea is “The absence of evidence does not constitute evidence of absence.” The take home message is that if you are relying on a direct test of your hypothesis (denoted H1), it is entirely possible that you will come to the wrong conclusion. Here is an example for illustration:
I grew up not far from the campus of Michigan State University. There is a place on campus where people can wade in the Red Cedar river and feed ducks. As a young girl, my family would often feed the ducks on warm summer weekends. Sadly, there would come a time each year when the ducks would disappear (autumn). My hypothesis was that, in late autumn, a man gathers the ducks and brings them to a warehouse where they overwinter. Then, in the summer, he puts the ducks back out on the river.
My hypothesis fit all of the available observations: The ducks disappeared in the autumn and came back each summer. There were often people driving around in maintenance trucks and many warehouses nearby in which the ducks could be stored. As a test of H1, I might decide to count the number of ducks present each day or the frequency, times and distance at which each maintenance truck passed the river. An analysis of these data might reveal that only certain maintenance trucks and personnel were likely duck moving candidates, leading me to further refine my hypothesis about the type of truck necessary. Alternately, I might find that the ducks do not, in fact, disappear in late autumn. Rather, their numbers slowly decrease starting around the time that students arrive on campus. This might lead me to hypothesize a link between the density of students and duck disappearances. I would be on the wrong track.
The correct way to approach H1 is to design a test that can disprove H0. In this case, H0 could be that there is no man who has duck gathering responsibilities or perhaps that ducks do not overwinter in warehouses. As a test, I might decide to check warehouses in the winter for duck sign or put a camera out on the Red Cedar river to film the duck herder in action. If I was unable to find evidence of a warehouse full of ducks or a man gathering them up in the autumn, H0 stands and confidence in H1 should diminish. Finally, let’s say that I did find a warehouse full of ducks. In this case, H0 has been disproven and confidence in H1 should be increased (Yay!) Note, however, that this evidence does not prove H1 as someone might be able to suggest a better explanation for the presence of ducks in warehouses.
In this era of disinformation and conspiracy theories, critical thinking skills are essential and testing the null hypothesis is an important tool. This is especially true in the case of hypotheses developed around anecdotal evidence as it can safeguard against confirmation bias. Recommender engines deliver results that confirm H1. We need to teach and devise technologies that encourage us to test H0.