< Back to Volume 14, Number 1
Volume 14: Number 1: Article 6
Publication Bias: The "File-Drawer Problem" in Scientific Inference
Jeffrey D. Scargle, Space Science Division National Aeronautics and
Space Administration, Ames Research Center, MS 245-3, NASA-Ames Research
Center, Moffett Field, CA, 94035-1000
Publication bias arises whenever the probability that a study is published
depends on the statistical significance of its results. This bias, often
called the file-drawer effect since the unpublished results are imagined
to be tucked away in researchers' file cabinets, is potentially a severe
impediment to combining the statistical results of studies collected
from the literature. With almost any reasonable quantitative model for
publication bias, only a small number of studies lost in the file-drawer
will produce a significant bias. This result contradicts the well known
Fail Safe File Drawer (FSFD) method for setting limits on the potential
harm of publication bias, widely used in social, medical and psychic
research. This method incorrectly treats the file drawer as unbiased,
and almost always misestimates the seriousness of publication bias.
A large body of not only psychic research, but medical and social science
studies, has mistakenly relied on this method to validate claimed discoveries.
Statistical combination can be trusted only if it is known with certainty
that all studies that have been carried out are included. Such certainty
is virtually impossible to achieve in literature surveys.
Keywords: publication bias, meta-analysis, file drawer effect, statistics
FULL TEXT:
Publication Bias: The "File-Drawer" Problem
in Scientific Inference
To purchase back issues contact Allen Marketing & Management: 1-800-627-0629
