There is a logical flaw in the statistical methods used across
experimental science. This fault is not a minor academic quibble: it
underlies a reproducibility crisis now threatening entire disciplines.
In an increasingly statistics-reliant society, this same deeply rooted
error shapes decisions in medicine, law, and public policy with profound
consequences. The foundation of the problem is a misunderstanding of
probability and its role in making inferences from observations.
Aubrey Clayton traces the history of how statistics went astray,
beginning with the groundbreaking work of the seventeenth-century
mathematician Jacob Bernoulli and winding through gambling, astronomy,
and genetics. Clayton recounts the feuds among rival schools of
statistics, exploring the surprisingly human problems that gave rise to
the discipline and the all-too-human shortcomings that derailed it. He
highlights how influential nineteenth- and twentieth-century figures
developed a statistical methodology they claimed was purely objective in
order to silence critics of their political agendas, including eugenics.
Clayton provides a clear account of the mathematics and logic of
probability, conveying complex concepts accessibly for readers
interested in the statistical methods that frame our understanding of
the world. He contends that we need to take a Bayesian approach--that
is, to incorporate prior knowledge when reasoning with incomplete
information--in order to resolve the crisis. Ranging across math,
philosophy, and culture, Bernoulli's Fallacy explains why something
has gone wrong with how we use data--and how to fix it.
