black black

Ch 3.1: Linear Regression

Lecture 4 - CMSE 381

Michigan State University
::
Dept of Computational Mathematics, Science /span> Engineering

Wed Jan 21, 2026

Announcements

Last time:

Covered in this lecture

Section 1

Simple Linear Regression

Setup

Y \approx β_{0} + β_{1} X

” $\approx$ ” .... “is approximately modeled as”

Setup

Y \approx β_{0} + β_{1} X

”

\approx

” .... “is approximately modeled as”

Example

Least squares criterion: Setup How do we estimate the coefficients?

Least squares criterion: RSS

sales \approx β_{0} + β_{1} TV

Residual sum of squares RSS is

\begin{array}{l} 𝑅𝑆𝑆 & = e_{1}^{2} + \dots + e_{n}^{2} \\ = \sum_{i} {(y_{i} - {\hat{β}}_{0} - {\hat{β}}_{1} x_{i})}^{2} \end{array}

Least squares criterion

Find

β_{0}

and

β_{1}

that minimize the RSS.

Least squares coefficient estimates

min_{β_{0}, β_{1}} \sum_{i} {(y_{i} - {\hat{β}}_{0} - {\hat{β}}_{1} x_{i})}^{2}

\begin{array}{l} \frac{∂𝑅𝑆𝑆}{\partial {\hat{β}}_{0}} & = - 2 \sum_{i} (y_{i} - {\hat{β}}_{0} - {\hat{β}}_{1} x_{i}) = 0 \\ \frac{∂𝑅𝑆𝑆}{\partial {\hat{β}}_{1}} & = - 2 \sum_{i} x_{i} (y_{i} - {\hat{β}}_{0} - {\hat{β}}_{1} x_{i}) = 0 \end{array}

\begin{array}{l} {\hat{β}}_{1} & = \frac{\sum_{i = 1}^{n} (x_{i} - \bar{x}) (y_{i} - \bar{y})}{\sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{2}} \\ {\hat{β}}_{0} & = \bar{y} - {\hat{β}}_{1} \bar{x} \end{array}

Coding group work

Next time

Next time:

Screenshot of the course schedule
from lectures 1 to 10.

Announcements