Tuesday, November 29, 2011
Are Most Published Findings False?
The problems are evidently ones of small sample size and publication bias. John Ioannadis has been arguing as much for some time (here is an essay at The Atlantic by David H. Freedman about Ioannidis' work). I am not versed in statistics but the basic arguments are easy to follow. With small sample sizes it's easy to find an effect due to chance which is only overturned after the study is replicated and the results fail to hold. Publication bias refers to the tendency of journals to favor interesting results (a positive result for a new hypothesis--or possibly a null result for a confirmed hypothesis) over non-interesting results for publication. Researchers are also less apt to report uninteresting null results in the first place.