From likelihood to posterior to MCMC
One running example (you observed k successes in n trials) shown two ways: the likelihood curve and its peak (the MLE), and a live Metropolis sampler whose draws pile up, one tick at a time, into the marginal posterior. In between, a two-parameter growth model shows what a likelihood surface looks like and how a profile likelihood is read off it.
1. The likelihood and the MLE
2. Two parameters: the likelihood surface and its profile
Panel 1 had one parameter, so the likelihood was a curve. Give the model two parameters and the likelihood becomes a surface, a hill standing over a map of every pair of parameter values. This panel uses the Von Bertalanffy growth model from the lab, fit to all 100 fish in vonbert.csv: length $= L_\infty(1 - e^{-\kappa\,\text{age}})$ plus normal noise. The map's two directions are the growth coefficient $\kappa$ and the asymptotic length $L_\infty$, and the height is the log-likelihood, measured down from the peak. The top of the hill is the MLE.
A profile is a walk along the ridge. Say you care about $\kappa$ and $L_\infty$ is a nuisance. To profile $\kappa$:
- Fix $\kappa$. Pick one value. On the map, that is a straight line running across the hill in the $L_\infty$ direction.
- Climb. Walk along that line to its highest point, which is the best $L_\infty$ for this $\kappa$. Record the height.
- Repeat. Do that for every $\kappa$. The highest points trace the orange ridge path over the hill, and their heights, plotted against $\kappa$, are the profile likelihood.
The 95% profile interval is every $\kappa$ whose ridge height is within 1.92 of the peak, the same rule as panel 1. The mistake the lab warns about is to skip step 2: freeze $L_\infty$ at one value and walk a straight line in the $\kappa$ direction instead. That is a slice (blue). Because $\kappa$ and $L_\infty$ trade off against each other, the hill is a tilted, diagonal ridge, and a straight slice cuts across it. The slice falls away faster than the ridge does, so its interval is too narrow, and if the frozen value is wrong it peaks in the wrong place too.
The noise SD $\sigma$ is a third parameter. To draw a surface in three dimensions, $\sigma$ is set to its best value at every point on the map, which is itself a profile over $\sigma$. The profile you see for $\kappa$ is therefore the full three-parameter profile, the one confint() returns in Exercise M2.
3. A Metropolis sampler builds the marginal posterior
What the sampler does, one step at a time. It wanders over values of $p$, and the values it visits pile up into the posterior. Each step:
- Propose. From the current $p$, jump to a nearby candidate $p' = p + \text{(a random nudge of size "step")}$.
- Score. Compute the posterior ratio $r = \dfrac{\text{posterior}(p')}{\text{posterior}(p)}$. Because the unknown normalizing constant cancels, this is just $\dfrac{p'^{\,k}(1-p')^{\,n-k}}{p^{\,k}(1-p)^{\,n-k}}$, likelihood × (flat) prior.
- Accept or reject. If the candidate is more probable ($r \ge 1$), move there. If it is less probable, move there only with probability $r$ (flip a biased coin); otherwise stay put and record the current value again.
That is the entire algorithm. Downhill moves are sometimes accepted, which is what lets the chain explore the whole posterior instead of just climbing to the peak. Run it and watch the histogram fill in the exact posterior curve without ever computing the integral in Bayes' rule.
The lower panel is the marginal posterior of $p$: take every draw the chain has made, sort them into bins, and that shape is your answer. The orange tick on the axis marks where the newest draw landed, so you can watch single draws turning into a distribution. With one parameter the marginal posterior is the whole posterior; in the two-parameter version the marginals are shadows of a joint cloud, and reporting one means throwing the other parameter away.
Things to try
- The peak is the MLE. Drag p in panel 1 to the top of the likelihood curve. It lands on k/n.
- The 1.92 rule. Tick the log-likelihood box. Every p whose log-likelihood is within 1.92 of the peak is inside the 95% profile interval (1.92 is half of the 3.84 chi-square cutoff). Set k = 3, n = 10 and compare that lopsided interval with the symmetric Wald interval printed beside it.
- A profile walks the ridge. In panel 2, press Side view along the nuisance. Looking straight along the $L_\infty$ axis, the outline of the hill is exactly the orange profile curve below it, because every point on that outline is the highest point along one line of sight. Then press Top down: the profile interval is the shadow the dashed 1.92 contour casts on the $\kappa$ axis, and the slice interval is only the stretch of the blue line inside that contour.
- Freezing a nuisance parameter misleads. Still in panel 2, drag Freeze L∞ up to 100, the round number the lab's grid search nailed it to. The slice now peaks at $\kappa = 0.207$ instead of the MLE, $0.241$, and its interval, 0.184 to 0.234, is 38% narrower than the profile interval and misses the MLE entirely. That is the lab's trap in miniature. Press Freeze at the MLE and the slice peaks in the right place but its interval is still too narrow. Only re-fitting $L_\infty$ at every $\kappa$ gets the width right.
- MCMC recovers the posterior. Hit Run and watch the teal histogram in panel 3 fill in the dashed black posterior curve. The MCMC never computes that curve. It just wanders and the samples pile up in the right shape.
- Read the interval off the draws. The shaded band is the middle 95% of the draws themselves, which is the credible interval you would report. Step slowly from Reset and watch it lurch about while the chain is short, then settle as the marginal fills in. Nothing here is computed from the analytic curve, which is the point: the same recipe works for posteriors that have no formula.
- Step size matters. Set the step very small: the chain crawls and the acceptance rate is high but mixing is slow. Set it very large: most proposals are rejected (low acceptance) and the chain gets stuck. A middling step (acceptance ≈ 20–45%) works best.
- More data → tighter posterior. Raise n (keeping k/n fixed) and both the likelihood and the posterior get narrower.