12 Pre-Class Assignment: Change of Basis#

Readings for this topic (Recommended in bold)#

Goals for today’s pre-class assignment#

Properties of Invertible Matrices
The Basis of a Vector Space
Change of Basis

%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import sympy as sym
sym.init_printing(use_unicode=True)

1. Review the Properties of Invertible Matrices#

Let $A$ be an $n \times n$ matrix. Recall that the following statements are equivalent, i.e., either all of them are true or all of them are false.

The column vectors of $A$ form a basis for $R^n$
$|A| \ne 0$
$A$ is invertible.
$A$ is row equivalent to $I_n$ (i.e. its reduced row echelon form is $I_n$)
The system of equations $Ax = b$ has a unique solution (for any $b$).
$rank(A) = n$

✅ DO THIS: Find a value of $c$ for which the following vectors do NOT form a basis for $\mathbb{R}^3$?

\[B = \{(1,2,4), (9,7,3), (2,c,8)\}\]

There are many correct ways to do this, but explain how you found the value of $c$.

Put your answer to the above question here.

Next, we will verify in many ways that for the value of $c$ that you found, the vectors do not form a basis.

The column vectors of $A$ do not form a basis for $\mathbb R^n$#

✅ DO THIS: Define a numpy matrix A whose columns are the vectors in $B$. Use the value of $c$ that you found in the previous question.

#Put your answer to the above question here

from answercheck import checkanswer

checkanswer.matrix(A,'eca9eadb119ffb3a3bc66bfd2916bdd9');

$|A| = 0$#

✅ DO THIS: From the above properties, if $B$ was a basis of $\mathbb{R}^3$ then $|A| \neq 0$, and if $B$ is not a basis for $\mathbb{R}^3$, then $|A| = 0$. Calculate and display the determinant of $A$.

#Put your answer to the above question here

$A$ is not invertible.#

✅ DO THIS: Since the determinant is zero we know that there isn’t an inverse to A. Try using python to calculate the inverse of $A$ and comment on what happens.

#Put your code for the above question here

Put your comment on what happens when you run the code here

$A$ is not row equivalent to $I_n$ (i.e. its reduced row echelon form is not $I_n$)#

✅ DO THIS: According to the property above the reduced row echelon form of an invertable matrix is the Identity matrix. Verify using the python sympy library and store the reduced row echelon matrix in a variable called rref if you really need to check it.

#put your answer to the above question here

from answercheck import checkanswer

checkanswer.matrix(rref,'ccb18925583aa87bb18d2681114b6f69');

The system of equations $Ax = b$ does not have a unique solution.#

If the columns of $A$ formed a basis, then for any vector $b$, the system of equations $Ax = b$ would have exactly one solution. Hence, if we can find a vector $b$ for which the system of equations $Ax = b$ has more than one solution (or no solutions), then the columns of $A$ do not form a basis.

✅ DO THIS:

Find at least two solutions $\vec{x}_1$ and $\vec{x}_2$ to $Ax=b$ for the vector $b=(12,13,15)$. Use code to verify that both of your solutions work. Store your solutions in variables called x1 and x2.

##put your code for the above question here

from answercheck import checkanswer

checkanswer.vector(A*x1,'f8450eed05f4539f0b2aed57dde95d2b');
checkanswer.vector(A*x2,'f8450eed05f4539f0b2aed57dde95d2b');

$\text{rank}(A) \neq n$#

The final property says that if the columns of $A$ form a basis, then the rank should equal the dimension of $\mathbb R^n$, i.e., $n$. So if $\text{rank}(A) \neq n$, then the columns of $A$ do not form a basis. Find a python function to calculate the rank of the matrix $A$. Display your answer.

#Put your answer to the above quesion here

✅ QUESTION: Without doing any calculations (i.e. only using the above properties), how many solutions are there to $Ax=0$? What is(are) the solution(s)?

Put your answer to the above question here.

2. The Basis of a Vector Space#

Let $U$ be a vector space with basis $B=\{u_1, \ldots, u_n\}$, and let $u$ be a vector in $U$. Because a basis “spans” the vector space, we know that there exists scalars $a_1, \dots, a_n$ such that:

\[ u = a_1u_1 + \dots + a_nu_n\]

Since a basis is a linearly independent set of vectors we know the scalars $a_1, \dots, a_n$ are unique.

The values $a_1, \dots, a_n$ are called the coordinates of $u$ relative to the basis ($B$) and is typically written as a column vector:

\[\begin{split} u_B = \left[ \begin{matrix} a_1 \\ \vdots \\ a_n \end{matrix} \right] \end{split}\]

We can create a transition matrix $P$ using the inverse of the matrix with the basis vectors being columns.

\[P = [ u_1 \ldots u_n ]^{-1}\]

Now we will show that matrix $P$ will transition vector $u$ in the standard coordinate system to the coordinates relative to the basis $B$:

\[ u_B = Pu\]

EXAMPLE: Consider the vector $u = \left[ \begin{matrix} 5 \\ 3 \end{matrix} \right]$ and the basis vectors $B = \{(1,2), (3,-1)\}$. The following code calculates the $P$ transition matrix from $B$ and then uses $P$ to calculate the values of $u_B$ ($a_1$ and $a_2$):

u = np.matrix([[5],[3]])
sym.Matrix(u)

B = np.matrix([[1,2],[3,-1]]).T
sym.Matrix(B)

P = np.linalg.inv(B)
ub = P*u

sym.Matrix(ub)

Here we would like to view this from $\mathbb R^n$. Let $$B=[u_1 \dots u_n],$$ then the values of $u_B$ can be found by solving the linear system $$u = B u_B.$$ The columns of $B$ are a basis, therefore, the matrix $B$ is a $n\times n$ square matrix and it has an inverse. Therefore, we can solve the linear system and obtain $$u_B = B^{-1} u = Pu.$$

Let’s try to visualize this with a plot:

ax = plt.axes();


#Blue arrow representing first Basis Vector
ax.arrow(0, 0, B[0,0],B[1,0], head_width=.2, head_length=.2, fc='blue', ec='blue');


#Green arrow representing Second Basis Vector
plt.plot([0,B[0,1]],[0,B[1,1]],color='green'); #Need this line to make the figure work. Not sure why.
ax.arrow(0, 0, B[0,1],B[1,1], head_width=.2, head_length=.2, fc='green', ec='green');

#Original point u as a red dot
ax.scatter(u[0,0],u[1,0], color='red');

plt.show()
#plt.axis('equal');

Notice that the blue arrow represents the first basis vector and the green arrow is the second basis vector in $B$. The solution to $u_B$ shows 2 units along the blue vector and 1 units along the green vector, which puts us at the point (5,3).

This is also called a change in coordinate systems.

✅ QUESTION: What is the coordinate vector of $u$ relative to the given basis $B$ in $\mathbb R^3$?

\[u = (9,-3,21)\]

\[B = \{(2,0,-1), (0,1,3), (1,1,1)\}\]

Store this coordinate in a variable ub for checking:

#Put your answer here

from answercheck import checkanswer

checkanswer.vector(ub,'f72f62c739096030e0074c4e1dfca159');

Let’s look more closely into the matrix $P$, what is the meaning of the columns of the matrix $P$?

We know that $P$ is the inverse of $B$, therefore, we have $$BP=I.$$ Then we can look at the first column of the $P$, say $p_{1}$, we have that $Bp_1$ is the column vector $(1,0,0)^\top$, which is exactly the first component from the standard basis. This is true for other columns.

It means that if we want to change an old basis $B$ to a new basis $B'$, we need to find out all the coordinates in the new basis for the old basis, and the transition matrix is by putting all the coordinates as columns.

Here is the matrix $B$ again:

B = np.matrix([[2,0,-1],[0,1,3],[1,1,1]]).T
sym.Matrix(B)

The first column of P should be the solution to $Bx=\left[ \begin{matrix} 1 \\ 0 \\ 0 \end{matrix} \right]$. We can use the numpy.linalg.solve function to find this solution:

# The first column of P should be 
u1 = np.matrix([1,0,0]).T
p1 = np.linalg.solve(B,u1)
p1

We can find a similar answer for columns $p_2$ and $p_3$:

# The second column of P should be 
u2 = np.matrix([0,1,0]).T
p2 = np.linalg.solve(B,u2)
p2

# The third column of P should be 
u3 = np.matrix([0,0,1]).T
p3 = np.linalg.solve(B,u3)
p3

# concatenate three column together into a 3x3 matrix
P = np.concatenate((p1, p2, p3), axis=1)
sym.Matrix(P)

# Find the new coordinate in the new basis
u = np.matrix([9,-3,21]).T
UB = P*u
print(UB)

This should be basically the same answer as you got above.

3. Change of Basis#

Watch the following video about change of basis.

from IPython.display import YouTubeVideo
YouTubeVideo("P2LTAUO1TdA",width=640,height=360, cc_load_policy=True)

Now consider the following two bases in $\mathbb R^2$:

\[B_1 = \{(1,2), (3,-1)\}\]

\[B_2 = \{(3,1), (5,2)\}\]

The transformation from the “standard basis” to $B_1$ and $B_2$ can be defined as the column vectors $P_1$ and $P_2$ as follows:

B1 = np.matrix([[1,2],[3,-1]]).T
P1 = np.linalg.inv(B1)

sym.Matrix(P1)

B2 = np.matrix([[3,1],[5,2]]).T
P2 = np.linalg.inv(B2)

sym.Matrix(P2)

✅ DO THIS: Find the transition matrix $T$ that will take points in the $B_1$ coordinate representation and put them into $B_2$ coordinates. NOTE this is analogous to the robot kinematics problem. We want to represent points in a different coordinate system.

# Put your answer to the above question here.

from answercheck import checkanswer

checkanswer.matrix(T,'dcc03ddff982e29eea6dd52ec9088986')

✅ QUESTION: Given $u_{B_1} = \left[ \begin{matrix} 2 \\ 1 \end{matrix} \right]$ (a point named $u$ in the $B_1$ coordinate system) and your calculated transition matrix $T$, what is the same point expressed in the $B_2$ basis (i.e. what is $u_{B2}$)? Store your answer in a variable named ub2 for checking.

ub1 = np.matrix([[2],[1]])
sym.Matrix(ub1)

##Put your code here

from answercheck import checkanswer

checkanswer.vector(ub2,'9a5fe29254c07cf59ebdffcaba679917')

There are three bases $B_1$, $B_2$, and $B_3$. We have the transition matrix $P_{12}$ from $B_1$ to $B_2$ and the transition matrix $P_{23}$ from $B_2$ to $B_3$. In $R^n$, we can compute the transition matrix as $$P_{12}=B_2^{-1}B_1,\quad P_{23}=B_3^{-1}B_2$$

Then we can find all other transition matrices. $$\begin{align*} P_{13} &= B_3^{-1}B_1=B_3^{-1}B_2*B_2^{-1}B_1= P_{23}P_{12}\\ P_{21} &= B_1^{-1}B_2 = (B_2^{-1}B_1)^{-1}=P_{12}^{-1}\\ P_{32} &= B_2^{-1}B_3 = (B_3^{-1}B_2)^{-1}=P_{23}^{-1}\\ P_{31} &= B_1^{-1}B_3 = (B_3^{-1}B_1)^{-1}=P_{13}^{-1}=(P_{23}P_{12})^{-1}=P_{12}^{-1}P_{23}^{-1} \end{align*}$$

The result is true for general vector spaces and can be extended to many bases.

Written by Dr. Dirk Colbry, Michigan State University, some modifications by Dr. Santhosh Karnik, Michigan State University
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CMSE/MTH 314 Course Website

12 Pre-Class Assignment: Change of Basis

Contents

12 Pre-Class Assignment: Change of Basis#

Readings for this topic (Recommended in bold)#

Goals for today’s pre-class assignment#

1. Review the Properties of Invertible Matrices#

The column vectors of \(A\) do not form a basis for \(\mathbb R^n\)#

\(|A| = 0\)#

\(A\) is not invertible.#

\(A\) is not row equivalent to \(I_n\) (i.e. its reduced row echelon form is not \(I_n\))#

The system of equations \(Ax = b\) does not have a unique solution.#

\(\text{rank}(A) \neq n\)#

2. The Basis of a Vector Space#

3. Change of Basis#