Class 5 · Likelihood and Parameter Estimation
STAT 517: Advanced Statistical Models · Fall 2026
This interactive companion follows the Class 5 notes. We check the two validity conditions numerically, trace counts to a mass function and counts to a density, match outcomes to supports before shapes, see the log-likelihood as a surface whose maximizer is the MLE, and finish by finding the same MLE six different ways.
Each WebR code box runs R directly in your browser. Press Run Code, inspect the result, change something, and run it again. Run the cells in order because later cells use objects created earlier. Checkpoint cells intentionally omit answers so they can be completed during class.
This page needs no add-on packages: everything below is base R. It should load faster than the earlier labs for that reason.
4.1 Setup
Keep every plot small. Multi-panel plots must stay inside a single cell, because the layout set by par() does not carry over from one cell to the next.
4.2 The two validity conditions
A mass function or density must be nonnegative everywhere, which you can read off any plot below, and must sum or integrate to 1 over its support. The second condition is the one worth checking numerically.
A density height is not a probability. A mass value is one.
4.3 From data to a distribution: discrete
Dividing counts by n alone makes the bar heights sum to 1. There is no bin width to divide by, because the support is a set of isolated points.
The Poisson support runs past 10, so summing only the plotted range falls a little short of 1.
4.4 From data to a distribution: continuous
Here the counts are divided by n times the bin width, so it is the bar area that becomes 1. Shrinking the bin width then gives the smooth density.
4.5 A gallery: support first, then shape
The support decides whether a distribution can describe an outcome at all, because it must contain every value the response can take. Shape separates whatever survives that test. Change the parameters and run it again.
Checkpoint. Name a plausible response distribution for each outcome: whether a patient relapses; the number of hospital visits; a strictly positive waiting time. For each one, say which feature of the outcome decided your choice, and name a distribution above that the same feature rules out. Use the cell to plot whichever candidates you want to compare.
4.6 The likelihood, and where it peaks
The log-likelihood is a function of the parameter with the data held fixed. With two parameters it is a surface, and the MLE is the point on the floor directly beneath its peak.
On the left the log-likelihood is a curve in \(\mu\) alone. On the right it is a surface in both parameters, shown as contours seen from above.
The dashed line and the dot mark the same thing: the maximizer.
4.7 Lab: one likelihood, six solvers
Every group fits the same normal model to the same twenty observations using an assigned solver, then the results are collected and compared. The MLE is defined by the likelihood, and nothing in that definition says how the value is found.
| Gp | Solver | Call |
|---|---|---|
| 1 | optim, Nelder–Mead |
optim(c(0, 0), nll) |
| 2 | optim, BFGS |
optim(c(0, 0), nll, method = "BFGS") |
| 3 | optim, CG |
optim(c(0, 0), nll, method = "CG") |
| 4 | nlm |
nlm(nll, c(0, 0)) |
| 5 | nlminb |
nlminb(c(0, 0), nll) |
| 6 | optim, L-BFGS-B |
optim(c(0, 1), nll_sigma, method = "L-BFGS-B", lower = c(-Inf, 1e-6)) |
Part 1. Run your assigned solver. Report \(\widehat\mu\), \(\widehat\sigma\), and the convergence code. Groups 1 to 5 must exponentiate the second parameter, since it is \(\log\sigma\).
Part 2. Compare with the closed forms \(\widehat\mu=\bar y\) and \(\widehat\sigma=\sqrt{\frac{1}{n}\sum_i (y_i-\bar y)^2}\), which in R are mean(y) and sqrt(mean((y - mean(y))^2)). Run the cell for the values your solver should be reproducing, then say how many decimal places you match to.
Part 3. Compute \(\widehat\sigma^2_{\text{MLE}}\) and var(y), then their ratio. Account for the number you get.
Part 4. We collect all six results. Do they agree, and to how many decimal places? Note that a convergence code means different things in different functions, so check the help page for the one you used: ?optim, ?nlm, or ?nlminb.
Part 5. Minimize nll_sigma with method = "BFGS" and no lower argument. Inspect what comes back before deciding whether to trust it.
Where this goes next
On September 3 we generalize the linear model itself. Choosing a response distribution and a link function turns the likelihood machinery of this class into the GLM family, and the binomial GLM with a logit link becomes simple logistic regression.