Source: Wikipedia
An article recently appeared in Nature which reports on a warning issued by statisticians about the misuse of p-values. The article focuses on two common problems regarding p-values: (1) Misinterpreting a p-value of .05 as indicating that there is a 95% chance that the test hypothesis is true; (2) Mistaking statistical significance as a measure of practical importance, or otherwise excessively relying upon p-values when engaged in critical thinking about the meaning and value of the results of a study.
However, the article itself risks confusion in two ways: First, by not clearly explaining the phrase 'at least as extreme as' when it gives the standard definition of a p-value of .05 (used as the criterion for statistical significance): "it signifies that if the null hypothesis is true, and all other assumptions made are valid, there is a 5% chance of obtaining a result at least as extreme as the one observed."
The second way in which the article risks confusion is by not clearly distinguishing between two methodological problems surrounding p-values: (1) There is widespread confusion about the meaning and implications of p-values; (2) There is widespread disagreement about the theoretical and practical importance of p-values when it comes to interpreting and evaluating the results of experiments or other scientific studies.
Nevertheless, it is refreshing and illuminating to see scientists engaged in this thought-provoking discussion about scientific methodology, and I hope it continues.
