Bayesian statistics

Introduction

Please note: a Statement of Participation is not issued for this course.

This free OpenLearn course, Bayesian statistics, is an extract from the Open University course M249 Practical modern statistics [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)] , a second level course that provides a broad perspective on modern statistics by introducing you to four important topics: medical statistics, time series, multivariate analysis and Bayesian statistics. Each of these is given a practical introduction with an emphasis on real problems and real data. Students are expected to have a good knowledge of statistical ideas and methods at an introductory level. Some basic mathematical skills are required. No knowledge of calculus is required.

Bayesian statistics consists of material from M249 Book 4, Bayesian statistics, and has three sections in total. You should set aside between three to four hours to study each of the sections; the whole extract should take about 12 hours to study. The extract is a small part (around 5%) of a large course that is studied over eight months, and so can give only an approximate indication of the level and content of the full course.

Some of the ideas which underpin Bayesian statistics are discussed in this extract and a framework for Bayesian inference is introduced. Bayesian methods have become extremely popular and are used in many diverse areas including medicine, criminal justice and internet search engines. Examples used to illustrate the ideas presented here reflect these diverse areas; this is typical of M249. It is relatively self-contained and should be reasonably easy to understand for someone who has not studied any of the previous texts in the course. However, a few techniques and definitions taught in earlier books in M249 are present in the extract without explanation. 

Mathematical/statistical content at the Open University is usually provided to students in printed books, with PDFs of the same online. This format ensures that mathematical notation is presented accurately and clearly. The PDF of this extract thus shows the content exactly as it would be seen by an Open University student. However, the extract isn't entirely representative of the module materials, because there are no explicit references to use of the M249 software or to video material (although please note that the PDF may contain references to other parts of M249). In this extract, some illustrations have also been removed due to copyright restrictions.

Regrettably, mathematical and statistical content in PDF form is not accessible using a screenreader, and you may need additional help to read these documents.

Section 1 discusses several ways of estimating probabilities, including simply calculating probabilities and using relative frequencies to estimate probabilities. For probabilities that cannot be estimated objectively, we also look at subjective estimation of probabilities.

Section 2 starts by reviewing ideas of conditional probabilities. A result at the heart of Bayesian statistics is then introduced, Bayes’ theorem. You will see how it can be used to update beliefs about a proposition when data are observed, or further information becomes available.

Section 3 introduces the main ideas of the Bayesian inference process. The prior distribution summarises beliefs about the value of a parameter before data are observed. The likelihood function summarises information about a parameter contained in observed data and the posterior distribution represents what is known about a parameter after the data have been observed.

This OpenLearn course is an adapted extract from the Open University course M249 Practical modern statistics.