12 In-Class Assignment: Change of Basis#

Alternative visual representation showing how the four Fundamental Subspaces spaces map to each other

Image from: wikipedia

%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import sympy as sym
sym.init_printing(use_unicode=True)

1. Review Pre-class assignment#

  • 12–Change_Basis_pre-class-assignment.ipynb

2. The Four Fundamental Spaces - Instructor Led#

In this course we represent a system of linear equations as \(Ax=b\). The matrix \(A\) can be viewed as taking a point \(x\) in the input space and projecting that point to \(b\) in the output space.

It turns out, everything we need to know about \(A\) is represented by four fundamental vector spaces. Two of the four spaces are easily defined as follows:

Row space of \(A\): linear combination of all rows of \(A\)

Column space of \(A\): linear combination of all columns of \(A\)

The other two fundamental spaces are defined by a concept called the Null Space. The Null space is calculated by finding all the solutions to the homogeneous system \(Ax=0\). The final two fundamental spaces are defined as follows:

Null space or kernel of \(A\): all \(x\) such that \(Ax=0\)

Null space of \(A^\top\): all \(y\) such that \(A^\top y =0\)

Classic picture of the four fundamental spaces. Please see text for detailed description.

Image from: https://kevinbinz.com/2017/02/20/linear-algebra/

The four fundamental subspaces#

  • Columnspace, \(\mathcal{C}(A)\)

  • Nullspace, \(\mathcal{N}(A)\)

  • Rowspaces, \(R(A)\)

    • All linear combinations of rows

    • All the linear combinations of the colums of \(A^\top\), \(\mathcal{C}(A^\top)\)

  • Nullspace of \(A^\top\), \(\mathcal{N}(A^\top)\) (the left nullspace of \(A\))

3. Matrix Representation of Vector Spaces#

Consider the following matrix \(A\).

\[\begin{split} \left[ \begin{matrix} 1 & 0 & 3 \\ 0 & 1 & 5 \\ 1 & 1 & 8 \end{matrix} \right] \end{split}\]

✅ QUESTION: What is the reduced row echelon form of \(A\)?

# Put your answer to the above question here.

from answercheck import checkanswer

checkanswer.matrix(rref,'1731818a1555cc33a778a4eb76af945c');

ROW SPACE The first and second (non zero) rows of the above matrix “spans” the same space as the orignal three row vectors in \(A\). We often call this the “row space” and it can be written as a linear combination of the non-zero rows of the reduced row echelon form:

\[row(A) = \{r(1,0,3)^\top+s(0,1,5)^\top\} = \mbox{span}((1,0,3)^\top,(0,1,5)^\top)\]

✅ DO THIS: Calculate the solutions to the system of homogeneous equations \(Ax=0\). This is often called the NULL SPACE or sometimes KERNEL of \(A\).

#Put your answer here

✅ DO THIS: We introduced two subspaces of \(\mathbb R^3\). Pick one vector from the row space of \(A\) and another vector from the null space of \(A\). Find the dot product of these two vector.

#Put your answer here

✅ Question: Did you get the same value for the dot product as your neighbor (or if you are working by yourself, did you get the same value for the dot product if you made different choices for the representatives of the row space and null space of \(A\))? Explain your answer.

Put your answer to the above question here

✅ DO THIS: What is the reduced row echelon form of \(A^T\)?

#Put your answer here

COLUMN SPACE: The first and second (non zero) rows of the above matrix “spans” the same space as the original three column vectors in \(A\). We often call this the “column space” (or “image space”) of \(A\) and it can be written as a linear combination of the non-zero rows of the reduced row echelon form of \(A^T\):

\[col(A) = \{a(1,0,1)^\top+b(0,1,1)^\top\} = \mbox{span}((1,0,1)^\top,(0,1,1)^\top)\]

✅ DO THIS: Calculate the solutions to the system of homogeneous equations \(A^Tx=0\). This is often called the NULL SPACE of \(A^T\).

✎ Do This - Erase the contents of this cell and replace it with your answer to the above question! (double-click on this text to edit this cell, and hit shift+enter to save the text)

Note: The fact that you got zero as the dot product for a vector from the row space and a vector from the null space was not a coincidence!

Theorem: Every vector \(x\) in the null space of \(A\) is perpendicular to every row of \(A\), because \(Ax=0\). The Null space and the Row space are orthogonal subspaces of \(\mathbb R^n\).

Every vector \(y\) in the null space of \(A^T\) is perpendicular to every column of \(A\), because \(A^Ty=0\).

Example #1#

Consider the following system of linear equations.

\[ x_1 - x_2 + x_3 = 3 \]
\[ -2x_1 + 2x_2 - 2x_3 = -6 \]

✅ DO THIS: What are the solutions to the above system of equations?

# Put your code here

✅ DO THIS: Come up with a specific arbitrary solution (any solution will do) to the above set of equations.

Put your answer to the above question here.

✅ DO THIS: Now consider only the left hand side of the above matrix and solve for the kernel (null Space) of A:

\[\begin{split} A = \left[ \begin{matrix} 1 & -1 & 1 \\ -2 & 2 & -2 \end{matrix} \right] \end{split}\]
#Put your answer here

✅ DO THIS: Express an arbitrary solution as the sum of an element of the kernel of the transformation defined by the matrix of coefficients and a particular solution.

Put your answer to the above question here.

✅ DO THIS: Discuss in your group and the class your solution from above. How does the solution to \(Ax=b\) relate to the solution to \(Ax=0\). If you were to plot all solutions, what shape does it take? How does this shape relate to the kernel?

Put your answer to the above question here.

3. Practice Nutrition#

Big Annie’s Mac and Cheese fans want to improve the levels of protein and fiber for lunch by adding broccoli and canned chicken. The nutrition information for the foods in this problem are

Nutrient

Mac and Cheese

Broccoli

Chicken

Shells and White Cheddar

Calories

270

51

70

260

Protein (g)

10

5.4

15

9

Fiber (g)

2

5.2

0

5
Logo for Annie's Mac and Cheese

She wants to achieve the goals with exactly 400 calories, 30 g of protein, and 10 g of fiber by choosing the combination of these three or four servings. (Assume that we can have non-integer proportions for each serving.)

✅ Question a: We consider all four choices of food together. Formulate the problem into a system of equations $\(Ax = b.\)\( Create your matrix \)A\( and the column vector \)b$ in np.matrix.

import numpy as np
#####Start your code here #####
A = np.matrix()
b = np.matrix()
#####End of your code here#####

✅ Question b: In this and next question, we only consider three out of the four choices. What proportions of these servings of the three foods (Mac and Cheese, Broccoli, and Chicken) should be used to meet the goal? (Hint: formulate it as a system of equations and solve it).

#Put your answer here

✅ Question c: She found that there was too much broccoli in the proportions from part (b), so she decided to switch from classical Mac and Cheese to Annie’s Whole Wheat Shells and White Cheddar. What proportions of servings of the new three food should she use to meet the goals?

#Put your answer here

✅ Question d: Based on the solutions to parts (b) and (c), what are the possible proportions of serving for the four food that meet the goal.

Put your answer here

✅ Question e: Solve the system of equations from part (a). You need to first decide the three outcomes: One solution, No solutions, Infinitely many solutions. Then for One solution, write down the solution; for Infinitely many solutions, write down all the solutions with free variables.

#Put your answer here

Put your answer here

Written by Dr. Dirk Colbry, Michigan State University Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.