Confidence Intervals for Expected Response & Prediction Intervals

STAT 306 - Module 05

Learning Objectives

By the end of this lecture, you should be able to:
- Distinguish between estimating the expected response \(\mathbb{E}[Y_{\text{new}}]\) and predicting a new individual observation \(Y_{\text{new}}\).
- State the sampling distribution and standard error of the fitted value \(\widehat{Y}_{\text{new}}\).
- Construct and interpret a confidence interval for the expected response at \(x_{\text{new}}\).
- Explain why confidence bands are narrowest at \(\bar{x}\) and widen away from the center (the “double-cone” shape).
- State the distribution and variance of the prediction error \(\widehat{Y}_{\text{new}} - Y_{\text{new}}\).
- Construct and interpret a prediction interval for a new observation at \(x_{\text{new}}\).
- Explain why prediction intervals are always wider than confidence intervals for the expected response.
- Compute and compare both intervals in R using predict().

Expected response

We know that \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\) is the predicted value of \(Y\) when \(X=x_{\text{new}}\).
\(\widehat{Y}_{\text{new}}\) is also an estimator for the mean value of the response when \(X=x_{\text{new}}\).
- This is called the expected response.

Now, we will construct a confidence interval for the expected response at a given value \(x_{\text{new}}\).
- Note that this is not an interval for a new observation when \(X=x_{\text{new}}\) (we will do this later).
- This is an interval for the mean value of \(Y\) when \(X=x_{\text{new}}\).

Activity: Constructing a CI for \(\mathbb{E}\left[Y_{\text{new}}\right]\)

Compute \(\mathbb{E}[\widehat{Y}_{\text{new}}]\).
Compute \(Var(\widehat{Y}_{\text{new}})\).
What is the sampling distribution of \(\widehat{Y}_{\text{new}}\)?
Construct a pivotal quantity for \(\widehat{Y}_{\text{new}}\).
Construct a confidence interval for \(\mathbb{E}[Y_{\text{new}}]\).

Hint: In Module 03 we calculated that \(Cov(\bar{Y},\widehat{\beta}_1) = 0\).

Constructing a CI for \(\mathbb{E}\left[Y_{\text{new}}\right]\): \(\mathbb{E}\left[\widehat{Y}_{\text{new}}\right]\)

The mean of \(\widehat{Y}_{\text{new}}\) is \[\begin{align} \mathbb{E}\left[\widehat{Y}_{\text{new}}\right] &= \mathbb{E}\left[\widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\right]\\ &= \mathbb{E}[\widehat{\beta}_0] + x_{\text{new}} \mathbb{E}[\widehat{\beta}_1]\\ &= \beta_0 + \beta_1 x_{\text{new}} \;. \end{align} \]

Constructing a CI for \(\mathbb{E}\left[Y_{\text{new}}\right]\): \(\text{Var}\left[\widehat{Y}_{\text{new}}\right]\)

\[ \begin{align} Var(\widehat{Y}_{\text{new}}) &= Var\left(\bar{Y} - \widehat{\beta}_1\bar{x} + \widehat{\beta}_1x_{\text{new}}\right) \\ &= Var\left(\bar{Y} + \widehat{\beta}_1(x_{\text{new}}-\bar{x})\right) \\ &= Var(\bar{Y}) + (x_{\text{new}}-\bar{x})^2Var(\widehat{\beta}_1) + 2(x_{\text{new}}-\bar{x})Cov(\bar{Y},\widehat{\beta}_1)\\ &= \frac{\sigma^2}{n} + \frac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} \\ &= \sigma^2\left[\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right] \;. \end{align} \]

iClicker 1

What is the distribution of the prediction \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\)?

\(N\left(0, \dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(N\left(\beta_0 + \beta_1 x_{\text{new}}, \dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(t\) with \(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\) degrees of freedom.
None of the above.

iClicker 1: Solution

What is the distribution of the prediction \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\)?

\(N\left(0, \dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(N\left(\beta_0 + \beta_1 x_{\text{new}}, \dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(t\) with \(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\) degrees of freedom.
None of the above.

Sampling Distribution of \(\widehat{Y}_{\text{new}}\)

Using \(\widehat{\beta}_0 = \bar{Y} - \widehat{\beta}_1\bar{x}\) and \(\widehat{\beta}_1 = \frac{\sum_{i=1}^n(x_i-\bar{x})Y_i}{\sum_{i=1}^n(x_i-\bar{x})^2}\), we can write \(\widehat{Y}_{\text{new}}\) directly in terms of the \(Y_i\)s: \[ \widehat{Y}_{\text{new}} = \bar{Y} + \widehat{\beta}_1(x_{\text{new}}-\bar{x}) = \sum_{i=1}^n \underbrace{\left[\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}\right]}_{c_i}Y_i \;. \]
The \(c_i\) are constants, so \(\widehat{Y}_{\text{new}}\) is a linear combination of the independent normal \(Y_i\)s, and hence normally distributed.
Therefore, \[\widehat{Y}_{\text{new}} \sim N\left(\beta_0 + \beta_1 x_{\text{new}}, \sigma^2\left[\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right]\right) \;.\]

Pivotal quantity for the expected response

Standardizing \(\widehat{Y}_{\text{new}}\), we have

\[\frac{\widehat{Y}_{\text{new}} - \mathbb{E}\left[Y_{\text{new}}\right]}{\sigma\sqrt{\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}} \sim N(0,1)\]

Similarly to the case of \(\widehat{\beta}_1\), replacing \(\sigma\) with \(\hat{\sigma}\), we obtain \[\frac{\widehat{Y}_{\text{new}} - \mathbb{E}\left[Y_{\text{new}}\right]}{\hat{\sigma}\sqrt{\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}} \sim t_{n-2} \;.\]

Finally,

\[\begin{align} CI(\mathbb{E}[Y_{\text{new}}], 1-\alpha) &= \widehat{Y}_{\text{new}} \pm t^*_{n-2,1-\frac{\alpha}{2}}\times \widehat{\text{SE}}(\widehat{Y}_{\text{new}})\\ &=\widehat{Y}_{\text{new}} \pm t^*_{n-2,1-\frac{\alpha}{2}}\times \hat{\sigma}\sqrt{\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}} \;. \end{align}\]

iClicker 2

Which of the following is the 95% confidence interval for \(\mathbb{E}[Y_{\text{new}}]\)?

\(\widehat{Y}_{\text{new}} \pm z^*_{0.975}\times \hat{\sigma}\sqrt{\dfrac{1}{n} + \dfrac{\bar{x}^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}\sqrt{\dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}^2\left(\dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \sigma\sqrt{\dfrac{1}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)

iClicker 2: Solution

Which of the following is the 95% confidence interval for \(\mathbb{E}[Y_{\text{new}}]\)?

\(\widehat{Y}_{\text{new}} \pm z^*_{0.975}\times \hat{\sigma}\sqrt{\dfrac{1}{n} + \dfrac{\bar{x}^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}\sqrt{\dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}^2\left(\dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \sigma\sqrt{\dfrac{1}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)

Confidence Interval for the Expected Response in R

We are \(95\%\) confident that the mean body mass of penguins with a flipper length of \(200 \text{ mm}\) is between \(4116\text{ g}\) and \(4201\text{ g}\).

Question

How does the width of the 95% confidence interval for \(\mathbb{E}\left[Y_{\text{new}}\right]\) change as \(x_{\text{new}}\) moves away from \(\bar{x}\)? What does this mean in terms of our uncertainty about \(\mathbb{E}\left[Y_{\text{new}}\right]\)?

Solution

\[ CI\big(\mathbb{E}\left[Y_{\text{new}}\right], 1-\alpha\big) = \widehat{Y}_{\text{new}} \pm t^*_{n-2,1-\frac{\alpha}{2}}\times \hat{\sigma}\sqrt{\frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}} \;.\]

By inspection of the confidence interval, the width increases as \(x_{\text{new}}\) moves away from \(\bar{x}\).
This implies that the farther \(x_{\text{new}}\) is from \(\bar{x}\), the less certainty we have for \(\mathbb{E}[Y_{\text{new}}]\) (or equivalently, the closer \(x_{\text{new}}\) is to \(\bar{x}\), the more certainty we have for \(\mathbb{E}[Y_{\text{new}}]\)). The confidence interval is narrowest at \(x_{\text{new}} = \bar{x}\).

Confidence Interval for the Expected Response

The width of the CI depends on \((x_{\text{new}}-\bar{x})^2\):
- it is narrowest at \(x_{\text{new}} = \bar{x}\)
- it widens as \(x_{\text{new}}\) moves away, producing a “double-cone” shape around the regression line.

Prediction Intervals

Predictions

A prediction is a guess of the value of a new observation \(Y_{\text{new}}\) at a new given value \(x_{\text{new}}\).
- A prediction \(\widehat{y}_i\) for a point \(x_i\) in our data is called a fitted value.

The regression line \(\widehat{Y} = \widehat{\beta}_0 + \widehat{\beta}_1 x\) is an estimate of the mean of \(Y\) for a given value \(x\).
The prediction \(\widehat{Y}_{\text{new}}\), which is an estimator for the mean of \(Y_{\text{new}}\), when \(X=x_{\text{new}}\), is the best predictor of \(Y_{\text{new}}\) (in terms of MSE).
- So \(\widehat{Y}_{\text{new}}=\widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\) is both an estimator for the conditional mean and a predictor of \(Y_{\text{new}}\).

Expected response vs prediction

Note that in the first part of this lecture we constructed a confidence interval for the expected response; i.e., \(\mathbb{E}[Y_{\text{new}}]\) when \(x=x_{\text{new}}\).
Now, we want to predict a particular response \(Y_{\text{new}}\) at \(x=x_{\text{new}}\).
- Naturally, most values of the response will be different from the mean response at that point (i.e., points are off the regression line).

Think of the difference between estimating the mean of a normal distribution versus predicting the value of a normal random variable.
- Also, what value would you use to predict the value of a normal random variable?

Prediction Intervals

A prediction interval is an interval that likely contains the value of a new observation \(Y_{\text{new}}\) at a given value \(x=x_{\text{new}}\).
This is different from a confidence interval for the expected response \(\mathbb{E}\left[Y_{\text{new}}\right]\) when \(x=x_{\text{new}}\).
- This is an interval of plausible values for the mean value of \(Y\) when \(x=x_{\text{new}}\).

When interpreting a prediction interval, we can say that there’s approximately a \((1-\alpha)100\%\) probability that a new observation \(Y_{\text{new}}\) at \(x=x_{\text{new}}\) falls inside the interval.
- This is because \(Y_{\text{new}}\) is still random after we compute the interval.
- Compare with the CI for \(\mathbb{E}[Y_{\text{new}}]\): the expected response is a fixed (unknown) number, so a computed CI either contains it or it doesn’t.
Prediction intervals are wider than confidence intervals for the expected response because they account for both the uncertainty in estimating the mean response and the variability of individual observations around that mean.

Prediction Intervals: Surreal Scenario

To understand the difference between prediction intervals and confidence intervals, let’s remove all the uncertainty in the model parameters by assuming that \(\beta_0\), \(\beta_1\), and \(\sigma^2\) are known.
- Never happens in practice, but useful for understanding the difference between the two types of intervals.

In this surreal scenario, the expected response \(\mathbb{E}\left[Y_{\text{new}}\right] = \beta_0 + \beta_1 x_{\text{new}}\) is a fixed, known value.
- There is no uncertainty about it (we are not using \(\widehat{\beta}_0\) and \(\widehat{\beta}_1\) to estimate it).
However, a new observation \(Y_{\text{new}} = \beta_0 + \beta_1 x_{\text{new}} + \varepsilon_{\text{new}}\) still has inherent variability due to the error term \(\varepsilon_{\text{new}}\).
- Remember: \(Y_{\text{new}} \sim N(\beta_0 + \beta_1 x_{\text{new}}, \sigma^2)\).

Therefore, even with known parameters, a prediction interval is given by: \[ (\beta_0 + \beta_1 x_{\text{new}}) \pm z^*_{1-\frac{\alpha}{2}} \times \sigma \] where \(z^*_{1-\frac{\alpha}{2}}\) is the \((1-\frac{\alpha}{2})\) quantile of the standard normal distribution.
Even with known parameters, we can only predict \(Y_{\text{new}}\) within an interval that accounts for this irreducible variability: \(\sigma\).

Prediction Intervals: Realistic Scenario

In practice, we don’t know the parameters \(\beta_0\), \(\beta_1\), and \(\sigma^2\).
- We estimate them from data, which introduces additional uncertainty.
Since \(\widehat{\beta}_0\) and \(\widehat{\beta}_1\) depend on the random sample, we have uncertainty about their true values.
- In other words, \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0+\widehat{\beta}_1x_{\text{new}} \neq \mathbb{E}\left[Y_{\text{new}}\right]\)!
- We need to account for the uncertainty in estimating the regression line.
- Therefore, we cannot just take the interval \((\beta_0 + \beta_1 x_{\text{new}}) \pm z^*_{1-\frac{\alpha}{2}} \times \sigma\).

Consider the prediction error \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\).

Activity

Goal: In this activity, we will work towards constructing a prediction interval:

Step 1: Show that \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\) is normally distributed.
Step 2: Obtain the mean of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\).
Step 3: Obtain the variance of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\).
Step 4: Pivotal quantity for \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\).
Step 5: Construct a prediction interval for \(Y_{\text{new}}\) at \(x=x_{\text{new}}\).

iClicker 3

What is \(\text{Var}\left(\widehat{Y}_{\text{new}} - Y_{\text{new}}\right)\)?

\(\sigma^2\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} - \sigma^2\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} + \sigma^2\)
\(\dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\)

iClicker 3: Solution

What is \(\text{Var}\left(\widehat{Y}_{\text{new}} - Y_{\text{new}}\right)\)?

\(\sigma^2\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} - \sigma^2\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\)
\(\dfrac{\sigma^2}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} + \sigma^2\)
\(\dfrac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\)

iClicker 4

As \(n\to\infty\), what happens to the variance of the prediction error, \(\text{Var}\left(\widehat{Y}_{\text{new}} - Y_{\text{new}}\right)\)?

Tends to zero.
Tends to \(\infty\).
Does not change.
Decreases but does not tend to zero.
Increases but does not tend to \(\infty\).

iClicker 4: Solution

As \(n\to\infty\), what happens to the variance of the prediction error, \(\text{Var}\left(\widehat{Y}_{\text{new}} - Y_{\text{new}}\right)\)?

Tends to zero.
Tends to \(\infty\).
Does not change.
Decreases but does not tend to zero.
Increases but does not tend to \(\infty\).

Step 1: Distribution Family of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\)

As we saw before, \(\widehat{Y}_{\text{new}} = \sum_{i=1}^n c_iY_i\), where \(c_i = \frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}\) are constants.
Therefore, \[ \widehat{Y}_{\text{new}}-Y_{\text{new}} = c_1Y_1 + \dotsb + c_nY_n - Y_{\text{new}} \] is a linear combination of \(Y_1,\dotsc,Y_n,Y_{\text{new}}\).
By the model assumptions, \(Y_1,\dotsc,Y_n,Y_{\text{new}}\) are independent normal random variables, so \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\) is normally distributed.

Step 2: \(\mathbb{E}\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right]\)

The mean of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\) is: \[ \begin{align} \mathbb{E}\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right] &= \mathbb{E}\left[\widehat{\beta}_0+\widehat{\beta}_1x_{\text{new}}-Y_{\text{new}}\right]\\ & = \mathbb{E}\left[\widehat{\beta}_0+\widehat{\beta}_1x_{\text{new}}\right] - \mathbb{E}[Y_{\text{new}}]\\ & = \beta_0 + \beta_1x_{\text{new}} - \beta_0 - \beta_1x_{\text{new}}\\ & = 0 \end{align} \]

Step 3: \(\text{Var}\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right]\)

The variance of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\) is:

\[ \text{Var}\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right] = \text{Var}\left(\widehat{Y}_{\text{new}}\right) + \text{Var}\left(Y_{\text{new}}\right) - 2\text{Cov}\left(\widehat{Y}_{\text{new}},Y_{\text{new}}\right) \]

But \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\) is based on the data used to fit the model, while \(Y_{\text{new}}\) is a new observation independent of the data used to fit the model.
- Therefore, \(Cov(\widehat{Y}_{\text{new}}, Y_{\text{new}}) = 0\).
Then, \(\text{Var}\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right] = \text{Var}\left(\widehat{Y}_{\text{new}}\right) + \text{Var}\left(Y_{\text{new}}\right)\)
- We have a variability term for estimating the mean response: \(Var\left(\widehat{Y}_{\text{new}}\right)\)
  - This can go to zero by increasing the accuracy of our estimates (i.e., collecting more data: \(n\to\infty\)).
- And we have a variability term for the new observation: \(Var\left(Y_{\text{new}}\right)\)
  - The variability term for the new observation is the irreducible error \(\sigma^2\).
Finally, \[ \begin{align} Var\left[\widehat{Y}_{\text{new}}-Y_{\text{new}}\right] &= Var\left(\widehat{Y}_{\text{new}}\right) + Var\left(Y_{\text{new}}\right)\\ &= \frac{\sigma^2}{n} + \frac{(x_{\text{new}}-\bar{x})^2\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} + \sigma^2 \\ &= \sigma^2\left[1 + \frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right] \;. \end{align} \]

Step 4: Standardized version of \(\widehat{Y}_{\text{new}}-Y_{\text{new}}\)

Therefore, we have that \[\frac{\widehat{Y}_{\text{new}}-Y_{\text{new}}}{\sigma\sqrt{1 + \frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}} \sim N(0,1) \;.\]
Similarly to what we did for \(\widehat{\beta}_1\), we can replace \(\sigma\) with \(\hat{\sigma}\) to obtain a pivotal quantity: \[\frac{\widehat{Y}_{\text{new}}-Y_{\text{new}}}{\hat{\sigma}\sqrt{1 + \frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}} \sim t_{n-2} \;.\]

Step 5: The prediction interval

Finally, a \((1-\alpha)100\%\) prediction interval for a new observation \(Y_{\text{new}}\) at \(X=x_{\text{new}}\) is given by:

\[ \begin{align} PI(Y_{\text{new}}, 1-\alpha) &= \widehat{Y}_{\text{new}} \pm t^*_{n-2,1-\frac{\alpha}{2}}\times \widehat{\text{SE}}_{\text{pred}} \\ &= \widehat{Y}_{\text{new}} \pm t^*_{n-2,1-\frac{\alpha}{2}}\times \hat{\sigma}\sqrt{1 + \frac{1}{n} + \frac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}} \;. \end{align} \]

iClicker 5

What is the 95% prediction interval for \(Y_{\text{new}}\) at \(x = x_{\text{new}}\)?

\(\widehat{Y}_{\text{new}} \pm z^*_{0.975}\times \hat{\sigma}\sqrt{1 + \dfrac{1}{n} + \dfrac{\bar{x}^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}\sqrt{1 + \dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}^2\left(1 + \dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \sigma\sqrt{1 + \dfrac{1}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)

iClicker 5: Solution

What is the 95% prediction interval for \(Y_{\text{new}}\) at \(x = x_{\text{new}}\)?

\(\widehat{Y}_{\text{new}} \pm z^*_{0.975}\times \hat{\sigma}\sqrt{1 + \dfrac{1}{n} + \dfrac{\bar{x}^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}\sqrt{1 + \dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \hat{\sigma}^2\left(1 + \dfrac{1}{n} + \dfrac{(x_{\text{new}}-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)\)
\(\widehat{Y}_{\text{new}} \pm t^*_{0.975}\times \sigma\sqrt{1 + \dfrac{1}{\sum_{i=1}^n(x_i-\bar{x})^2}}\)

Example

Let’s use the fitted model model <- lm(body_mass_g ~ flipper_length_mm, data = penguins_clean):
- Fit a linear regression model to predict body mass (g) using flipper length (mm).
- Construct a 95% confidence interval for the expected body mass when flipper length is 200 mm.
- Construct a 95% prediction interval for a new penguin’s body mass when flipper length is 200 mm.
Note that the predict function expects a data frame with the new \(x\) value and the same variable name as in the model.

In summary, when predicting in R, we can use the predict() function with the argument interval="confidence" for the confidence interval for the expected response, and interval="prediction" for the prediction interval.

Confidence vs Prediction Intervals

Both bands are narrowest at \(\bar{x}\), but the prediction band is always wider: it also includes the variability of a single new observation around the line.

iClicker 6

Compared to the 95% confidence interval for the expected response \(\mathbb{E}[Y_{\text{new}}]\), the 95% prediction interval for \(Y_{\text{new}}\) is…

roughly the same width.
narrower.
wider.
any of the above depending on the context.

iClicker 6: Solution

Compared to the 95% confidence interval for the expected response \(\mathbb{E}[Y_{\text{new}}]\), the 95% prediction interval for \(Y_{\text{new}}\) is…

roughly the same width.
narrower.
wider.
any of the above depending on the context.

iClicker 7

The prediction interval has an irreducible variability, meaning that no matter what the sample size is, we cannot reduce the variability more than this value. What value is this?

\(\text{Var}(\varepsilon)\)
\(\text{Var}(\bar{Y})\)
\(\text{Var}(\widehat{\beta}_1)\)
\(\text{Var}(\widehat{Y})\)
None of the above.

iClicker 7: Solution

The prediction interval has an irreducible variability, meaning that no matter what the sample size is, we cannot reduce the variability more than this value. What value is this?

\(\text{Var}(\varepsilon)\)
\(\text{Var}(\bar{Y})\)
\(\text{Var}(\widehat{\beta}_1)\)
\(\text{Var}(\widehat{Y})\)
None of the above.

What We Learned

Two Inferential Goals at \(X = x_{\text{new}}\):
- Expected Response: Estimate the mean response \(\mathbb{E}\left[Y_{\text{new}}\right] = \beta_0 + \beta_1 x_{\text{new}}\).
- New Observation: Predict an individual future observation \(Y_{\text{new}} = \beta_0 + \beta_1 x_{\text{new}} + \varepsilon_{\text{new}}\).
- Both goals share the same point estimate: \(\widehat{Y}_{\text{new}} = \widehat{\beta}_0 + \widehat{\beta}_1 x_{\text{new}}\).
Standard Errors: \[ \begin{aligned} \widehat{\text{SE}}(\widehat{Y}_{\text{new}}) &= \widehat{\sigma}\sqrt{\frac{1}{n} + \frac{(x_{\text{new}} - \bar{x})^2}{\sum_{i=1}^n (x_i - \bar{x})^2}} \quad \xrightarrow{n \to \infty} 0 \\ \widehat{\text{SE}}_{\text{pred}} &= \widehat{\sigma}\sqrt{1 + \frac{1}{n} + \frac{(x_{\text{new}} - \bar{x})^2}{\sum_{i=1}^n (x_i - \bar{x})^2}} \quad \xrightarrow{n \to \infty} \sigma \end{aligned} \]
Confidence vs. Prediction Intervals:
- Confidence Interval (\(\mathbb{E}[Y_{\text{new}}]\)): \(\widehat{Y}_{\text{new}} \pm t^*_{n-2, 1-\alpha/2} \times \widehat{\text{SE}}(\widehat{Y}_{\text{new}})\).
- Prediction Interval (\(Y_{\text{new}}\)): \(\widehat{Y}_{\text{new}} \pm t^*_{n-2, 1-\alpha/2} \times \widehat{\text{SE}}_{\text{pred}}\).
- A prediction interval is always wider because it accounts for individual variability \(\sigma^2\) (the “\(+1\)” under the radical).
- A PI can be read as a probability statement about \(Y_{\text{new}}\), which stays random after we compute the interval. A computed CI cannot, since \(\mathbb{E}[Y_{\text{new}}]\) is a fixed number.
Shape of the Bands:
- Both bands are narrowest at \(x_{\text{new}} = \bar{x}\) widening away from \(\bar{x}\) due to \((x_{\text{new}} - \bar{x})^2\).
Computation in R:
- predict(model, newdata, interval = "confidence") for the expected response.
- predict(model, newdata, interval = "prediction") for a new observation.