Ch 3.2: Multiple Linear Regression

Lecture 6 - CMSE 381

Michigan State University
::
Dept of Computational Mathematics, Science /span> Engineering

Mon, Jan 26, 2026

Announcements

Last time:

3.1 (Simple) linear regression

Announcements:

Homework #2 Due Sunday 2/8

Covered in this lecture

Multiple linear regression
Hypothesis test in that case
Forward and Backward Selection

Section 1

Review from last time

Linear Regression with One Variable

scatter plot with linear fit and residuals

Predict $Y$ on a single variable $X$

Y \approx β_{0} + β_{1} X

Find good guesses for

{\hat{β}}_{0}

,

{\hat{β}}_{1}

.

ŷ_{i} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{i}

e_{i} = y_{i} - ŷ_{i}

is the

i

th residual

residual sum of squares RSS

= \sum_{i} e_{i}^{2}

Evaluating the model

Linear regression is unbiased
Variance of linear regression estimates:
$\begin{array}{l} SE ({\hat{β}}_{0}) & = σ^{2} [\frac{1}{n} + \frac{{\bar{x}}^{2}}{\sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{2}}] \\ SE {({\hat{β}}_{1})}^{2} & = \frac{σ^{2}}{\sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{2}} \end{array}$
where $σ^{2} = Var (𝜀)$
Estimate $σ$ : ${\hat{σ}}^{2} = \frac{𝑅𝑆𝑆}{n - 2}$

The 95% confidence interval for $β_{1}$ approximately takes the form

{\hat{β}}_{1} \pm 2 \cdot SE ({\hat{β}}_{1})

Hypothesis test:

$β_{1} = 0$
$β_{1} \neq 0$
Test statistic $t = \frac{{\hat{β}}_{1} - 0}{SE ({\hat{β}}_{1})}$

Assessing the accuracy of the model

Residual standard error (RSE):

\begin{array}{l} 𝑅𝑆𝐸 & = \sqrt{\frac{1}{n - 2} 𝑅𝑆𝑆} \end{array}

R squared:

\begin{array}{l} R^{2} & = \frac{𝑇𝑆𝑆 - 𝑅𝑆𝑆}{𝑇𝑆𝑆} = 1 - \frac{𝑅𝑆𝑆}{𝑇𝑆𝑆} \end{array}

𝑇𝑆𝑆 = \sum_{i} {(y_{i} - \bar{y})}^{2}

Section 2

Multiple Linear Regression

Setup

Y = β_{0} + β_{1} X_{1} + β_{2} X_{2} + \dots β_{p} X_{p} + 𝜀

scatter plot in 3D of Y
versus X1 and X2. A planar
fit is shown with residuals
for each data point.

Estimation and Prediction

Given estimates

{\hat{β}}_{0}, {\hat{β}}_{1}, {\hat{β}}_{2}, \dots, {\hat{β}}_{p}

, prediction is

ŷ = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + \dots + {\hat{β}}_{p} x_{p}

Advertising data set example

𝚂𝚊𝚕𝚎𝚜 = β_{0} + β_{1} \cdot 𝚃𝚅 + β_{2} \cdot 𝚛𝚊𝚍𝚒𝚘 + β_{3} \cdot 𝚗𝚎𝚠𝚜𝚙𝚊𝚙𝚎𝚛

scatter plot in 3D of Y
versus X1 and X2. A planar
fit is shown with residuals
for each data point.

Table showing the
coefficients for a linear
regression of Sales versus
TV, radio, and newspaper
advertising all in 1000s of
dollars.

Interpretation of coefficients

𝚂𝚊𝚕𝚎𝚜 = β_{0} + β_{1} \cdot 𝚃𝚅 + β_{2} \cdot 𝚛𝚊𝚍𝚒𝚘 + β_{3} \cdot 𝚗𝚎𝚠𝚜𝚙𝚊𝚙𝚎𝚛

Single regression vs multi-regression

Sales versus TV advertising scatter plot with
linear fit line

Sales versus radio advertising scatter plot with
linear fit line

Sales versus newspaper advertising scatter plot
with linear fit line

scatter plot in 3D of Y versus X1 and
X2. A planar fit is shown with
residuals for each data point. Table
showing the coefficients for a linear
regression of Sales versus TV, radio,
and newspaper advertising all in
1000s of dollars.

Correlation matrix Correlation matrix
for TV, radio, newspaper, and sales
all in units of 1000 dollars.