Hierarchical Bayes, latent states, and management quantities — in normal English

What we did today, in one paragraph

Hierarchical models in plain English

A hierarchical model is a model in which some of the parameters themselves have parameters. The textbook example: you measured something at 30 sites, and you want a separate intercept $\alpha_j$ for each site, but you also think the sites are not totally unrelated — they all come from the same regional population. You write:

and then you estimate both the individual $\alpha_j$'s and the higher-level $\mu_\alpha$ and $\sigma_\alpha$ from the data. That's it. You have just acknowledged that sites are different but related, and the model will use that fact wisely.

The same logic applies to slopes, variances, intercepts on log scales, intercepts on logit scales, dispersion parameters — anything that varies across some grouping. The hierarchical machinery is reusable.

What "shrinkage" really means

Imagine you sampled 50 streams for steelhead presence. In 49 streams you have 30 observations each. In one stream you have a single observation, which happened to be a detection. If you treat that one stream as its own universe (no pooling), your estimated probability of presence in stream 50 is $1/1 = 1.0$.

If you pool partially, the model says: "Across all the streams I've seen, presence rates are roughly $\mu = 0.4$ with SD $\sigma = 0.2$. Stream 50 has one observation that says 'present.' Both pieces of information are valid. Let me weight them by their precision."

The result: stream 50's estimated occupancy gets pulled from 1.0 toward 0.4, ending up somewhere around 0.5 or 0.6. That is shrinkage. It is the same logic a human expert would apply: don't take one observation at face value when you have a population of similar streams to compare against.

Latent state models: separating what's happening from what you saw

In ecology, we almost never measure the quantity we care about. We don't see "true occupancy" — we see whether a camera or eDNA sample flagged a species during one visit. We don't see "true abundance" — we see a count that is a noisy fraction of the truth. The actual biological state is latent (hidden).

A latent-state model has two layers:

Estimating both layers at once is the whole point. If you ignored the observation layer and treated raw counts as truth, you would underestimate abundance, overestimate trend rates, and miss extinctions that are really just non-detections. The Bayesian latent-state framework gives you a clean place to write down both layers.

The two canonical examples

• Dynamic occupancy: $z[i, t]$ is the true occupancy of site $i$ in year $t$, evolving with persistence $\phi$ and colonization $\gamma$. The data are repeated detections $y[i, j, t]$ given $z$ and detection $p$. We met this in section E of the lab.
• N-mixture: $N_i$ is the true abundance at site $i$, drawn from Poisson($\lambda$). The data are repeated counts $y[i, j]$ drawn from Binomial($N_i, p$). We met this in section F.

Both models look complicated when you first see them, but each is just a process layer (one line) plus an observation layer (one line) plus priors. Once you internalise that split, dozens of other models in the literature suddenly become recognisable as variations on the same theme: capture-recapture, hidden Markov, dynamic N-mixture, false-positive occupancy, multistate models, etc.

Derived quantities — the Bayesian superpower for managers

A derived quantity is anything you compute from your model's parameters that you care about. Examples:

• $S_\text{msy} = b\sqrt{1/a} - b/a$ in a Beverton-Holt fit — spawners at maximum sustainable yield.
• The equilibrium occupancy $\gamma / (\gamma + (1 - \phi))$ in a dynamic occupancy model.
• The total population $\sum_i N_i$ across all sites in an N-mixture model.
• The probability that population $A$ has higher abundance than population $B$.

In maximum-likelihood land, you fit $(a, b)$, get point estimates and standard errors, and use the delta method or bootstrap to put a CI on $S_\text{msy}$. It works, but it's an approximation.

In Bayes, you compute $S_\text{msy}$ at every MCMC iteration, using that iteration's draw of $(a, b)$. The resulting set of values is the posterior distribution of $S_\text{msy}$ — with all the correlation between $a$ and $b$ correctly accounted for, no delta method, no bootstrap. You just write the derived-quantity line inside your nimbleCode block and ask MCMCsummary for it.

Why MCMC works (intuitively)

Bayes' rule tells you what the posterior is:

It does not tell you what shape the posterior has, or how to compute its mean, or how to sample from it. For all but a few toy problems, the right-hand side is so messy that no formula will hand you the answer.

Markov Chain Monte Carlo (MCMC) is a clever way to sidestep the messy denominator. The idea: build a random walk through parameter space that, in the long run, visits each region with probability proportional to the posterior. Tally up where the walk spent its time, and you have an approximation of the posterior.

The two most common engines are:

• Gibbs sampling: update each parameter from its full conditional distribution. Works great when those conditionals have known forms. JAGS and BUGS built their reputations on Gibbs.
• Metropolis-Hastings: propose a small random step, accept or reject based on the ratio of posterior densities. Works on essentially any model.

NIMBLE uses a mix of both (and more — slice samplers, block samplers, Hamiltonian Monte Carlo via add-ons) and lets you customise.

Why we switched JAGS → NIMBLE for the course

The model code is nearly identical — same distributions, same precision parameterisation, same logit/ilogit idiom. The reasons to switch are practical:

• NIMBLE is actively developed in R, with the maintainers responsive to bugs and feature requests.
• It compiles models to C++ for speed, then runs them as MCMC by default. You can also write your own samplers in R.
• It generalises: you can run MCMC, but also Sequential Monte Carlo, MCEM, Laplace approximation, or your own custom algorithms, on the same model object.
• The split between constants (sizes, indices, fixed covariates) and data (observed random variables) forces a clarity that JAGS' single-list approach often obscures.

The cheat sheet you need:

Common pitfalls (and how to avoid them)