Bayesian Reasoning

An Annotated Bibliography Compiled by Timothy McGrew

This brief annotated bibliography is intended to help students get started with their research. It is not a substitute for personal investigation of the literature, and it is not a comprehensive bibliography on the subject. For those just beginning to study Bayesian reasoning, I suggest the starred items as good places to start your reading.

L. Jonathan Cohen, An Introduction to the Philosophy of Induction and Probability (Oxford: Oxford University Press, 1989).

Full of interesting historical information and challenging suggestions. Cohen contrasts two major concepts of confirmation, which he terms "Baconian" and "Pascalian." The latter is the common probabilistic concept, but Cohen argues vigorously that Pascalian probability cannot be the whole story about the logic of confirmation.

J. Earman, ed., Testing Scientific Theories, vol. 10, Minnesota Studies in the Philosophy of Science (Minneapolis: University of Minnesota Press, 1983).

An excellent collection of essays, many pertaining to Glymour's work.

__________, Bayes or Bust? (Cambridge, MA: MIT Press, 1992).

A sympathetic but critical survey of the state of the art by a self-professed "lapsed Bayesian." Parts of this are fairly technical.

Clark Glymour, Theory and Evidence (Princeton: Princeton University Press, 1980).

Glymour's book caused quite a stir, largely because of two features: his novel "bootstrapping" approach (which can look circular at first glance) and his trenchant essay "Why I am not a Bayesian."

C. Howson and P. Urbach, Scientific Reasoning: the Bayesian Approach (La Salle, IL: Open Court, 1989).

The standard reference work for Subjective Bayesians. Howson and Urbach go well beyond presenting their position, giving detailed criticisms of alternatives. They are particularly critical of the classical tradition in statistical inference.

*P. Horwich, Probability and Evidence (Cambridge: Cambridge University Press, 1982).

An elegant and highly readable elementary treatment of the Bayesian approach to scientific reasoning. Horwich advocates a "degree of belief" approach to probability, but he rejects Subjective Bayesianism in favor of a "rationalist" construal in which an individual's probability assignments are subject to stronger constraints than mere coherence. He then applies the Bayesian methodology to many puzzles and problems and demonstrates its power. This one is well worth reading even if you don't accept all of his solutions.

R. Jeffrey, The Logic of Decision, 2nd ed. (Chicago: University of Chicago Press, 1983).

A standard textbook on Bayesian inference and decision theory from a Subjectivist or "Personalist" point of view. This book contains Jeffrey's own explication of "Jeffrey conditioning," a general probabilistic updating rule of which the standard Bayesian conditioning is merely a special case.

__________, Probability and the Art of Judgment (Cambridge: Cambridge University Press, 1992).

A collection of Jeffrey's essays applying and extending Subjective Bayesian methods. Some of the essays are technical: others are readable without a strong mathematical background so long as one has mastered the basic probability calculus.

H. Jeffreys, Scientific Inference (Cambridge: Cambridge University Press, 1937).

An early and forceful presentation of the Objective Bayesian point of view. Though Jeffreys is a high-powered writer and does not hesitate to invoke mathematics when it is required, there is much here that can be understood even by readers who lack strong mathematical preparation. The discussion of simplicity is particularly important.

__________, Theory of Probability, 2nd ed. (Oxford: Oxford University Press, 1948).

A historically important formulation of Objective Bayesianism. Jeffreys insists that in situations of complete ignorance, we must select a prior probability in such a way as to give experience the maximum impact on our posterior probabilities. In some cases this leads him to endorse "improper" priors that cannot be normalized. Often challenging reading, but rewarding.

H. Kyburg, Epistemology and Inference (Minneapolis: University of Minnesota Press, 1983).

Kyburg has long been one of the most vocal critics of Subjectivism in probability. This collection of his essays is indispensible for anyone who wants to see what can be said against Subjectivism. The essay on "Subjective Probability" is a classic. (Scan the subtitle for acronyms.)

James Logue, Projective Probability (Oxford: Oxford University Press, 1995).

Logue’s book develops a version of personalism and claims that it captures the univocal meaning of "probability." He is skeptical, however, about the attempt to resolve all questions about scientific inference by appealing to Bayesian conditionalization. The book contains a very interesting discussion of the problem of probabilistic "weight."

R. Miller, Fact and Method (Princeton: Princeton University Press, 1987).

Miller is deeply critical of Bayesian approaches to scientific reasoning. His exposition of Subjective Bayesianism is a model of clarity, and his criticisms, though uneven in quality, are sometimes exceedingly shrewd.

R. Rosenkrantz, Inference, Method and Decision: Towards a Bayesian Philosophy of Science (Dordrecht: D. Reidel, 1977).

The definitive treatise on Objective Bayesianism. Rosenkrantz advocates a the use of "maximum entropy" (or "maxent") priors. His discussion of the similarities and differences between various Objective Bayesian approaches is illuminating, and his treatment of simplicity and "sample coverage" merits close study. Of particular interest is Rosenkrantz's careful treatment of Popper's philosophy of science; he maintains that many of Popper's methodological insights can be recaptured within a Bayesian framework.

__________, "Why Glymour Is a Bayesian," in Earman (1983), pp. 69-98.

Just what it sounds like. Rosenkrantz argues that notwithstanding his criticisms of Bayesians, Glymour is actually more Bayesian than many who march under that banner.

*W. Salmon, The Foundations of Scientific Inference (Pittsburgh: University of Pittsburgh Press, 1966).

An early and trenchant argument for the applicability of Bayesian probability to problems of scientific inference. The initial survey of approaches to the problem of induction is very useful, and the Bayesian sections of the book are worth reading even though they are not the latest or deepest work on the subject.

__________, "Bayes's Theorem and the History of Science," in R. Stuewer, ed., Historical and Philosophical Perspectives of Science, vol. 5, Minnesota Studies in the Philosophy of Science (Minneapolis: University of Minesota Press, 1970), pp. 68-86.

A classic essay in which Salmon treats with good sense and sophistication the traditional problem of separating the context of discovery from the context of justification. In the end he argues for the applicability of Bayesian analyses to episodes in the history of science and suggests that the difficult problem of prior probabilities is best approached through "plausibility constraints."

__________, "Rationality and Objectivity in Science, or Tom Kuhn Meets Tom Bayes," in C. W. Savage, ed., Scientific Theories, vol. 14, Minnesota Studies in the Philosophy of Science (Minneapolis: University of Minnesota Press, 1990), pp. 175-204.

An attempt to clean up the subjectivity rampant in Kuhn's philosophy of science by imposing some Bayesian constraints on scientific inference. Kuhn, in his reply, disavowed even the constraints Salmon wanted to place on such reasoning, though it's not entirely clear whether he was caricaturing Salmon's position in so doing.

G. Schlesinger, The Sweep of Probability (Notre Dame: Notre Dame University Press, 1991).

A wide-ranging survey of the applications of probability theory in general and Bayesian methods in particular to various conundrums in epistemology and the philosophy of science.

A. Shimony, Search for a Naturalistic World View, vol. 1, Scientific Method and Epistemology (Cambridge: Cambridge University Press, 1993).

For the past several decades Shimony has made important contributions to Bayesian epistemology and philosophy of science. This collection of his papers contains much important material, including the initial essay endorsing "tempered" Bayesianism, the "Adamic derivation" of the probability calculus, and too many other goodies to list. Required reading for Bayesians (and non-Bayesians!) of any flavor.