Likelihood, priors, posteriors, and MCMC — in normal English

What we did today, in one paragraph

What is a likelihood?

A likelihood is a number — well, a function — that tells you how compatible your data are with a particular guess about the parameters. It uses the same formula as the probability of seeing the data, but it is read differently:

• Probability: Hold the parameter fixed, ask "how likely is each possible dataset?" Sum/integrate over datasets.
• Likelihood: Hold the data fixed (at what you actually observed), ask "how compatible is each possible parameter value with this dataset?" Do not sum/integrate over parameters — it isn't a probability over parameters.

What is a maximum-likelihood estimate?

The MLE is the parameter value that makes the data as plausible as possible. If you watched 50 cars and 30 ran the red light, the MLE for the probability of running the light is 30/50 = 0.6. If you weighed 100 fish and want $L_\infty$ for a Von Bertalanffy growth curve, the MLE is the value of $L_\infty$ that minimizes the squared distance between the curve and the points (when the noise is Gaussian — exactly the case where MLE and least-squares give the same answer).

For complicated models the MLE doesn't have a closed form. We use a computer to numerically optimize the log-likelihood. R's mle() and optim() functions do this for us; they also give back standard errors based on the curvature of the log-likelihood at the maximum (the second derivative — the "Hessian").

From prior to posterior, in pictures

Bayes does something the MLE cannot: it gives you a distribution over the parameter, not just a point. Three ingredients:

1. Prior $p(\theta)$: what you believed about $\theta$ before seeing the data. Often "I don't know much" — a wide, weakly informative distribution.
2. Likelihood $p(y \mid \theta)$: exactly the same likelihood as the morning.
3. Posterior $p(\theta \mid y)$: what you believe now. By Bayes' rule, $\text{posterior} \propto \text{prior} \times \text{likelihood}$.

If the prior is uniform and the likelihood is peaked, the posterior looks like the likelihood — and the posterior mode is the MLE. If the prior is strong, it pulls (or "shrinks") the posterior toward what you previously believed. As you get more data, the likelihood gets sharper and dominates: the prior becomes less and less important.

The big Bayesian flip

Frequentist statistics computes things like $P(\text{data} \mid H_0)$ — the famous p-value — and then asks you to translate that into a verdict on a hypothesis. Bayes computes the thing you actually wanted in the first place:

Notice that the Bayesian sentence puts the probability where you wanted it all along — on the hypothesis. That is the payoff for buying into the prior + posterior framework.

What does MCMC actually do?

For real models — anything with more than one or two parameters, or a non-conjugate likelihood — we cannot write the posterior down in closed form. Markov chain Monte Carlo is a clever algorithm that generates a long sequence of random samples whose long-run frequency is the posterior. You don't compute the posterior; you draw from it.

The simplest version (Metropolis):

1. Start somewhere.
2. Propose a small random move.
3. If the move improves the posterior, accept. If it makes the posterior worse, accept with some probability (the ratio of new to old posterior).
4. Otherwise stay put.
5. Repeat thousands of times. Throw out the first chunk ("burn-in"). The rest is a sample from the posterior.

Once you have those samples, every summary you want is a one-liner:

• posterior mean of $r$? mean(r_samples)
• 95% credible interval? quantile(r_samples, c(.025, .975))
• probability $r > 0.22$? mean(r_samples > 0.22)
• posterior of $K/2$? Compute $K/2$ for every sample of $K$ — that's the posterior of $K/2$.

Where does Nimble fit in?

You don't want to write Metropolis by hand for every new model. Nimble (and JAGS, Stan, etc.) take a BUGS-style description of your model — priors and likelihood — and automatically:

• build the computational graph,
• pick a sensible sampler for each parameter,
• run several independent chains from different starting values (nimbleMCMC() runs them one after another by default — true parallel execution takes extra setup),
• return posterior samples for whatever quantities you ask to monitor.

Why Nimble specifically (instead of JAGS)? Nimble's BUGS dialect is nearly identical to JAGS — most JAGS code runs unchanged inside a nimbleCode({...}) block — but it is much more extensible: you can write custom samplers, custom distributions, and compile to fast C++ all from R. Whole subfields (ecological state-space, capture-recapture, spatial models) have migrated to Nimble for this reason.

Common confusions to avoid

Looking ahead to Day 5

Tomorrow we take the Bayesian machinery you just learned and apply it to harder, more realistic models: hierarchical (random-effects) models, occupancy models, capture-recapture, and state-space models. Same Nimble syntax, deeper biology. The morning's maximum likelihood and the afternoon's conjugacy and logistic-growth fit you did today are the building blocks; everything tomorrow plugs together pieces you already understand.

Things to keep in your back pocket going into Day 5:

• How to write a nimbleCode block.
• The split between constants and data.
• What MCMCsummary and MCMCtrace tell you.
• How to compute any tail probability from posterior samples.
• The difference between a BCI and an HPDI.