Last updated: 2017-03-04
Code version: 5d0fa13282db4a97dc7d62e2d704e88a5afdb824
This illustrates how the prior, likelihood, and posterior behave for inference for a normal mean (\(\mu\)) from normal-distributed data, with a conjugate prior on \(\mu\).
Specifically the prior on \(\mu\) is N(\(\mu_0\), \(\tau_0^2\)) [dotted line] and the data is sampled from a normal distribution N(\(\mu\), \(\sigma^2\)), which gives the likelihood [black line]. Note that the likelihood is scaled so it fits nicely on the graph (remember, likelihoods only matter up to a constant, so you can scale them however is convenient).
Because the normal distribution is the conjugate prior for normal sampling, the posterior distribution is also a normal distribution, and is shown in red.
By Bayes theorem:
\[ \text{Pr}(\mu \, | \, \mathbf{y}, \sigma^2) \propto \text{Pr}(\mathbf{y} \, | \, \mu, \sigma^2) \text{Pr}(\mu) \]
\[ \text{N}(\mu_1, \tau_1^2) = \text{N}(\mu, \sigma^2) \text{N}(\mu_0, \tau_0^2) \]
where the posterior mean:
\[ \mu_1 = \frac{\frac{\mu_0}{\tau_0^2} + \frac{n \bar{y}}{\sigma^2}}{\frac{1}{\tau_0^2} + \frac{n}{\sigma^2}} \]
and the posterior variance:
\[ \tau_1^2 = (\frac{1}{\tau_0^2} + \frac{n}{\sigma^2})^{-1} \]
An interactive app that shows how the posterior distribution will change when the prior and the (scaled) data likelihood changes:
The source code of the app can be found here.
sessionInfo()R version 3.3.0 (2016-05-03)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.10.5 (Yosemite)
locale:
[1] en_NZ.UTF-8/en_NZ.UTF-8/en_NZ.UTF-8/C/en_NZ.UTF-8/en_NZ.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] tidyr_0.4.1 dplyr_0.5.0 ggplot2_2.1.0 knitr_1.15.1
[5] MASS_7.3-45 expm_0.999-0 Matrix_1.2-6 viridis_0.3.4
[9] workflowr_0.3.0 rmarkdown_1.3
loaded via a namespace (and not attached):
[1] Rcpp_0.12.5 git2r_0.18.0 plyr_1.8.4 tools_3.3.0
[5] digest_0.6.9 evaluate_0.10 tibble_1.1 gtable_0.2.0
[9] lattice_0.20-33 shiny_0.13.2 DBI_0.4-1 yaml_2.1.14
[13] gridExtra_2.2.1 stringr_1.2.0 gtools_3.5.0 rprojroot_1.2
[17] grid_3.3.0 R6_2.1.2 reshape2_1.4.1 magrittr_1.5
[21] backports_1.0.5 scales_0.4.0 htmltools_0.3.5 assertthat_0.1
[25] mime_0.5 colorspace_1.2-6 xtable_1.8-2 httpuv_1.3.3
[29] labeling_0.3 stringi_1.1.2 munsell_0.4.3 This site was created with R Markdown