Class 20 · Models for Zero-Inflated Counts

STAT 517: Advanced Statistical Models · Fall 2026

This interactive companion follows Chapter 14 and the reviewed class20_zero_inflated_counts.R. The fish example shows why a Poisson model cannot produce enough zeros and fits zero-inflated Poisson (ZIP) and zero-inflated negative binomial (ZINB) models; the PhD articles example is the in-class lab.

Press Run Code on each box in order. Later boxes reuse earlier objects, and you can edit a box and run it again. The fish data file is included, the articles data come with the pscl package, and pscl loads with the page (it provides zeroinfl()). Question 14.1 matches the handout; it is the in-class lab, whose group assignments and submission template are posted where the instructor hosts the lab threads.

Goals

• Recognize zero-inflated count data and explain why a Poisson model underpredicts the zeros.
• Write a zero-inflated model as a mixture of a degenerate distribution at 0 and a count distribution, with \(P(y_i = 0) = \pi_i + (1-\pi_i) f(0 \mid \lambda_i)\) and \(E(y_i \mid \mathbf{x}_i) = (1-\pi_i)\lambda_i\).
• Fit ZIP and ZINB models with pscl::zeroinfl(), using different covariates in the two parts.
• Compare count models with AIC and with observed versus expected zeros.
• Interpret count-part coefficients as incidence rate ratios and zero-part coefficients as odds ratios for a structural zero, with confidence intervals.
• Predict the mean count and the probability of each count for new covariates.

Part 1 · Zero-inflated counts

1. Fish caught by visitors to a state park

For 250 groups that visited a state park, we record how many fish the group caught (count), the number of children (child) and of people (persons) in the group, and whether the group used live bait (livebait) and brought a camper (camper). Some visitors do not fish, but there is no data on whether a group fished. Some groups that did fish caught nothing, so there are zeros both from groups that did not fish and from groups that fished but caught none.

142 of the 250 groups caught zero fish, while a few caught dozens.

2. The Poisson model’s missing zeros

Fit a Poisson regression first, as with any count analysis, and check it with the goodness-of-fit test. Then add up \(P(y_i = 0)\) over the 250 fitted Poisson means: that is the number of zeros the model expects.

The Poisson distribution does not fit, and the model expects only about 99 zeros: it cannot produce 142 zeros together with the large catches.

Part 2 · The zero-inflated mixture

3. Structural and sampling zeros

A zero-inflated model assumes that, with probability \(\pi\), \(y\) comes from a degenerate distribution at 0 (a group that does not fish), and with probability \(1 - \pi\) from a count distribution \(f(y \mid \lambda)\), Poisson for ZIP or negative binomial for ZINB:

\[ P(y = 0) = \underbrace{\pi}_{\text{structural}} + \underbrace{(1-\pi) f(0 \mid \lambda)}_{\text{sampling}}, \qquad P(y = k) = (1-\pi) f(k \mid \lambda), \quad k \ge 1. \]

Every nonzero probability shrinks by the factor \(1 - \pi\), and the mean is \((1-\pi)\lambda\). In the plots, the orange part of the bar at 0 is the structural zeros and the blue part the sampling zeros. Change pi.zero, lambda, or theta and run the box again.

With the same \(\pi\) and \(\lambda\), the ZINB count part is more spread out and right-skewed, and it produces more sampling zeros: \(P(y = 0)\) is \((2/7)^2 = 0.082\) under NB(5, 2), against \(e^{-5} = 0.007\) under Poisson(5).

4. Two links, two sets of coefficients

The zero part uses the logit link and the count part the log link, each with its own intercept and covariates:

\[ \log\frac{\pi_i}{1-\pi_i} = \delta_0 + \delta_1 u_{i1} + \cdots + \delta_q u_{iq}, \qquad \log \lambda_i = \beta_0 + \beta_1 v_{i1} + \cdots + \beta_r v_{ir}. \]

In zeroinfl(count ~ v1 + v2 | u1 + u2, data = ..., dist = "poisson"), the covariates to the left of | model the mean \(\lambda\) of the count part and those to the right model the probability \(\pi\) of a structural zero; without |, the same covariates enter both parts. dist = "negbin" gives the ZINB model. predict() returns the overall mean \((1-\hat\pi)\hat\lambda\) with type = "response", \(\hat\lambda\) with "count", \(\hat\pi\) with "zero", and \(P(Y = 0), P(Y = 1), \ldots\) with "prob".

Part 3 · ZIP and ZINB models for the fish data

5. ZIP with every covariate in both parts

AIC is \(-2\log L + 2k\), where \(k\) counts every estimated parameter: the coefficients in both parts, including both intercepts, plus \(\theta\) for a ZINB model. Here \(k = 10\), five coefficients in each part; the Poisson model has \(k = 5\). AIC() counts them this way.

The ZIP model fits far better than the Poisson model, and it expects about 142 zeros, matching the 142 observed.

6. Different covariates in the two parts

ZIP models are easier to interpret if they have fewer parameters. Let only child and camper affect the mean of the count (Poisson) part and only persons affect the probability of definitely catching zero fish.

All covariates are significant conditionally on the others in the reduced model (all Wald p-values < 0.05).

WarningFewer parameters is not a better fit

The reduced ZIP model has AIC 2073.2, higher than both the ZIP model with all covariates (1444.8) and the Poisson model (1594.0), because dropping livebait and persons from the count part costs a lot of likelihood. It is used here to show different covariates in the two parts; when the goal is the best-fitting model, compare AICs before simplifying.

7. Interpret both parts

Exponentiate and subtract 1. Rows starting count_ give the percent change in the mean count among groups that fish; rows starting zero_ give the percent change in the odds of a structural zero. For zeroinfl() fits, confint() gives Wald intervals.

AMONG the groups that are inclined to fish, we expect one additional child to be associated with a 64.75% decrease in (or 0.353 times) the mean number of fish caught, after controlling for whether or not the group has a camper; we are 95% confident that the decrease is between 57.12% and 71.03%. Groups with a camper are expected to catch 130.26% more fish (95% CI: 91.65% to 176.63% more), after controlling for number of children. One additional person is associated with 43.13% lower odds of definitely catching zero fish, that is, of belonging to the degenerate distribution at zero (95% CI: 21.73% to 58.68% lower).

8. Predicted mean catches

For every possible combination of children, camper, and group size, predict() gives the overall predicted mean \((1-\hat\pi)\hat\lambda\). Children are counted among the persons, so a group of \(p\) persons has at most \(p\) children, and the 1- and 2-person lines stop at 1 and 2 children. The last point on the 1-, 2-, and 3-person lines is a group made up only of children; no such group occurs in the data (every group has at least one adult), so those predictions are extrapolations.

The predicted mean decreases with every additional child, more steeply for groups with a camper. Larger groups are predicted to catch more fish (the curves do not cross), and groups with a camper more than groups without one.

9. Structural and sampling zeros for one group

newgroup2 is a group of 4 persons, including 2 children, without a camper.

\(P(Y = 0) = \hat\pi + (1-\hat\pi)e^{-\hat\lambda} = 0.2769 + 0.3913 = 0.6682\): about 28% of such groups are predicted not to fish and 39% to fish but catch nothing.

10. The ZINB model for the fish data

Changing dist = "poisson" to dist = "negbin" keeps the same covariates in each part.

The ZINB model’s AIC is far lower: \(\hat\theta = 0.373\) is small, so the catches of the groups that fish are strongly overdispersed. The count-part rate ratios keep their signs and similar sizes, but zero_persons changes the most: once the count part can produce many zeros by itself, fewer zeros must be attributed to the structural-zero process. Under the ZINB model, one additional person is associated with 81.11% lower odds of a structural zero, with a much wider interval (28.48% to 95.01% lower).

NoteWhat is the Log(theta) row?

A ZINB summary lists Log(theta) with the count-part coefficients. It is the estimate of \(\log\theta\), where \(\theta\) is the negative binomial parameter of the count part, \(\text{var} = \lambda + \lambda^2/\theta\); it is not a covariate. R estimates it on the log scale because \(\theta > 0\), and prints \(\hat\theta\) below the tables (Theta = 0.373 here). Its Wald test checks \(\theta = 1\), not “no overdispersion” (that would be \(\theta \to \infty\)), so it is not a test of ZINB against ZIP: judge overdispersion by the size of \(\hat\theta\) and by comparing AICs.

11. Observed versus expected zeros

The ZIP model with all covariates matches the 142 observed zeros; the Poisson model is far short.

Part 4 · In-class lab: articles by biochemistry PhD students

12. Load the data and choose a model

The bioChemists data in pscl record, for 915 biochemistry PhD students (Long, 1990), the number of articles art produced in the last 3 years of the PhD, with gender (fem), marital status (mar), the number of children under 6 (kid5), the prestige of the PhD program (phd), and the number of articles by the student’s mentor in the same period (ment). 275 students produced no articles.

TipQuestion 14.1 (in-class lab): Choose, interpret, and predict with a ZINB model

1. Choose. Using the goodness-of-fit test and the AIC output, which of the Poisson, ZIP, and ZINB models fits best? What does each step, from Poisson to ZIP and from ZIP to ZINB, add to the model?
2. Interpret. Interpret the count-part rate ratios for kid5 and ment and the zero-part odds ratio for ment, with their 95% intervals, the way the femWomen sentence does. Say which group of students each statement describes.
3. Predict. Student 1 is a married man with 2 children whose mentor published 1 article. Use predict(new.phd.zinb, newdata = student1, type = ...) with "count", "zero", "response", and "prob". Report \(\hat\lambda\), \(\hat\pi\), the predicted mean, and \(P(Y = 0)\). Why is \(P(Y = 0)\) much larger than \(\hat\pi\)? Verify \(P(Y = 0) = \hat\pi + (1-\hat\pi)\bigl(\hat\theta/(\hat\lambda + \hat\theta)\bigr)^{\hat\theta}\), with \(\hat\theta\) = new.phd.zinb$theta.

13. The reduced ZINB model

To improve interpretability, keep fem, kid5, and ment for the mean of the count part and only ment for the probability of a structural zero.

Model sentence, for femWomen: among the PhD students who are inclined to write articles (the non-degenerate distribution), after controlling for number of children and number of articles published by the PhD mentor, we expect female students to produce, on average, 20.36% fewer mean articles than male students in their final 3 years of PhD study; we are 95% confident that they produce between 8.37% and 30.8% fewer.

14. Predict for student 1

15. Worked example: student 2

For a single woman with no children whose mentor published 20 articles:

The predicted mean is 2.124 articles, and 1 article is the most likely outcome: the probabilities of 0, 1, 2, and 3 articles are 20.89%, 24.81%, 20.20%, and 13.93%. With a highly productive mentor, the probability of a structural zero is essentially 0, so these zeros are sampling zeros.

Check your understanding

• Zero-inflated counts have more zeros than a Poisson or negative binomial model with the same covariates predicts; compare observed and expected zeros, not only the goodness-of-fit test.
• A zero-inflated model mixes a degenerate distribution at 0, with probability \(\pi_i\), and a count distribution with mean \(\lambda_i\): \(P(y_i = 0) = \pi_i + (1-\pi_i) f(0 \mid \lambda_i)\) and \(E(y_i \mid \mathbf{x}_i) = (1-\pi_i)\lambda_i\).
• The zero part uses the logit link and the count part the log link; each has its own covariates and intercept. ZIP uses a Poisson count part and ZINB a negative binomial one.
• Count-part coefficients give incidence rate ratios for the mean among units that can produce counts; zero-part coefficients give odds ratios for a structural zero. Neither is the effect on the overall mean \((1-\pi_i)\lambda_i\).
• Choose among Poisson, ZIP, and ZINB with AIC() on the same responses; fewer parameters do not guarantee a better fit.
• predict() gives the overall mean, \(\hat\lambda\), \(\hat\pi\), and the probability of each count.