Class 2 · Simple Linear Regression

STAT 517: Advanced Statistical Models · Fall 2026

This interactive companion follows the Class 2 notes. We fit and interpret a simple linear regression, examine all five standard diagnostic plots, compare the original and log-response models, review slope inference, and distinguish confidence intervals from prediction intervals.

Each WebR code box runs R directly in your browser. Press Run Code, inspect the result, change something, and run it again. Checkpoint cells intentionally omit answers so they can be completed during class.

2.1 Model and least squares

For predictor (X) and continuous response (Y), the simple linear regression model is

\[ Y_i=\beta_0+\beta_1x_i+\varepsilon_i, \qquad E(\varepsilon_i)=0, \qquad \operatorname{Var}(\varepsilon_i)=\sigma^2. \]

The fitted line minimizes the residual sum of squares. The following small example lets you inspect fitted values and residuals directly.

Question 2.1: Connect the formulas

Why does the fitted least-squares line with an intercept pass through $(\bar x,\bar y)$? Why does estimating an intercept and slope leave $n-2$ residual degrees of freedom?

2.2 Advertising and sales

The marketing data contain 200 observations. Advertising budgets and sales are recorded in thousands of dollars. This page preloads the datarium package for the WebR browser session.

Fit mean sales using YouTube advertising budget.

Question 2.2: Interpret the fitted slope

Interpret the fitted YouTube slope in the units of the data. What additional information would be needed before giving it a causal interpretation?

2.3 Regression diagnostics

Each diagnostic plot asks a different question. The default plot(youtube.mod1) display omits which = 4, so we request all five numbered plots explicitly.

Use the plot number and axis labels when describing evidence:

Plot 1 checks the linear mean and residual pattern.
Plot 2 checks whether residual quantiles are approximately normal.
Plot 3 checks whether residual spread is approximately constant.
Plot 4 displays Cook’s distance.
Plot 5 combines standardized residuals and leverage.

Diagnostic checkpoint

Which diagnostic concerns are visible? Distinguish an unusual response from an unusual predictor value, and explain why a point can have high leverage without having a large residual.

2.4 Transforming the response

The scale-location plot suggests that the residual spread may not be constant. A log transformation is one possible response, but it changes both the model target and the slope interpretation. It should be judged using the complete model context and all diagnostics, not selected from one plot alone.

The fitted log-response slope is approximately $0.00316$. Use the code below to investigate the relevant changes in log-sales and proportional sales.

Question 2.3: Interpret a log-response slope

Classify each statement as true or false and justify your answer.

1000 * b1 represents a 3.16 increase in log-sales.
1000 * (exp(b1) - 1) represents a $3.16 increase in sales.
exp(b1) - 1 represents approximately a 0.316% increase in sales.
b1 represents a 0.00316 increase in log-sales when sales are measured in dollars.
b1 represents a 0.00316 increase in log-sales when sales are measured in thousands of dollars.

Changing the response unit from thousands of dollars to dollars adds a constant to the logged response. Use the fitted coefficients to see what changes.

2.5 Inference for the slope

For $H_0:\beta_1=0$, the slope $t$ statistic divides the estimated slope by its standard error. In simple linear regression with one predictor, the overall regression test satisfies $F=t^2$.

Question 2.4: Read the evidence in context

Explain what the slope estimate, $t$ statistic, overall $F$ statistic, $p$-value, and 95% confidence interval each contribute. Which give equivalent tests here? Which conclusions do not follow from these summaries?

2.6 Confidence and prediction intervals

The built-in cars data relate vehicle speed to stopping distance. A confidence interval targets the mean stopping distance at a specified speed. A prediction interval targets one new stopping distance and is wider because it also includes individual variation around the mean.

2.7 Training and testing extension

The 2024 script also held out 20 rows as test.dat and fitted car.mod2 using the remaining 30 rows. When test.dat is passed to predict(), only its speed column supplies predictor values; its observed dist values do not affect the fitted model or intervals.

The original script plotted the held-out observations but did not calculate a test-performance measure. The next cell extends it with test RMSE and empirical prediction-interval coverage.

Training/test checkpoint

Why was test.dat$dist not used by predict()? What would repeating the split under many random seeds reveal about the sensitivity of the fitted model and its test performance?

2.8 Takeaways

A slope must be interpreted with the units and study design.
Residual, Q-Q, scale-location, Cook’s-distance, and leverage plots answer different diagnostic questions.
A transformation changes the model target and interpretation.
In simple linear regression, the two-sided slope test and overall regression test are equivalent because $F=t^2$.
Prediction intervals include individual variation and are wider than confidence intervals for the mean response.
Held-out responses can evaluate predictions, but they must not be used to fit the training model.

Next class (August 25): multiple linear regression, categorical predictors, conditional associations, and model comparison.

2.9 Answer key

Checkpoint answers are intentionally omitted from this class-demo companion. The lecture notes provide the answer key for review after class discussion.