There is a logical flaw in the statistical methods used across
experimental science. This fault is not a minor academic quibble: It
underlies a reproducibility crisis now threatening entire disciplines.
In an increasingly statistics-reliant society, this same deeply rooted
error shapes decisions in medicine, law, and public policy, with
profound consequences. The foundation of the problem is a
misunderstanding of probability and its role in making inferences from
observations.
Aubrey Clayton traces the history of how statistics went astray,
beginning with the groundbreaking work of the 17th-century mathematician
Jacob Bernoulli and winding through gambling, astronomy, and genetics.
Clayton recounts the feuds among rival schools of statistics, exploring
the surprisingly human problems that gave rise to the discipline and the
all-too-human shortcomings that derailed it. He highlights how
influential 19th- and 20th-century figures developed a statistical
methodology they claimed was purely objective in order to silence
critics of their political agendas, including eugenics.
Clayton provides a clear account of the mathematics and logic of
probability, conveying complex concepts accessibly for listeners
interested in the statistical methods that frame our understanding of
the world. He contends that we need to take a Bayesian approach - that
is, to incorporate prior knowledge when reasoning with incomplete
information - in order to resolve the crisis. Ranging across math,
philosophy, and culture, Bernoulli's Fallacy explains why something
has gone wrong with how we use data - and how to fix it.
