Linear Algebra

Vector Spaces

↩ Back
01

Vector Spaces and Examples

Concept
The Eight Axioms

For any $u, v, w \in V$ and scalars $\lambda, \rho$, a vector space must satisfy:


1, 2$u+v = v+u$ and $(u+v)+w = u+(v+w)$
3, 4There is a zero vector $0$ with $u+0=u$, and each $u$ has an opposite $-u$ with $u+(-u)=0$
5, 6$\lambda(u+v) = \lambda u + \lambda v$ and $(\lambda+\rho)u = \lambda u + \rho u$
7, 8$\lambda(\rho u) = (\lambda\rho)u$ and $1u = u$

In $\mathbb{R}^n$, addition and scalar multiplication act component by component, and the zero vector is $0 = (0,\dots,0)^\top$.

Interactive
Addition and scalar multiplication in R²
u blue · v pink · result gold
2.0
1.0
-1.0
2.0
1.5
u + v = (1.0, 3.0)
Concept
Typical vector spaces
RnTuples $u=(u_1,\dots,u_n)^\top$, added and scaled component by component. The usual arrows of physics and geometry.
PnPolynomials $p(t) = a_0 + a_1t + \cdots + a_nt^n$ of degree at most $n$, with real coefficients $a_i$.

Both satisfy the same eight axioms, so every result proved for a general vector space $V$ applies to both at once.

Example
Zero vector and negative in Pn
1
The zero vector of $P_n$ is the zero polynomial, $p(t) = 0$, so $a_i = 0$ for every $i$.
2
Given $p(t) = a_0 + a_1t + \cdots + a_nt^n$, its negative is $-p(t) = -a_0 - a_1t - \cdots - a_nt^n$, also in $P_n$.
3
Sum and scalar multiple are defined coefficient by coefficient, exactly as in $\mathbb{R}^{n+1}$, so $P_n$ behaves exactly like $\mathbb{R}^{n+1}$.
02

Subspaces

Concept
Subspace

A subset $H \subset V$ is a subspace of $V$ when it satisfies three conditions.


1The zero vector of $V$ is in $H$.
2$H$ is closed under addition: for any $u,v \in H$, $u+v \in H$.
3$H$ is closed under scalar multiplication: for any $u \in H$ and scalar $\lambda$, $\lambda u \in H$.

Condition 1 is technically redundant, since it follows from condition 3 with $\lambda = 0$, but it is worth checking first: it is the fastest way to rule out a candidate.

All three must hold for $H$ to be a subspace; failing just one is enough to rule it out.

Interactive
Does this line pass through the origin?
H = {(x, x + c)}
0.8
(0, 0) is not in H: not a subspace
Method
Two ways to prove H is a subspace
Axioms
Check the zero vector is in $H$, then that $H$ is closed under addition and scalar multiplication directly, using the definition of $H$.
Span
Write an arbitrary element of $H$ as a linear combination of fixed vectors, $H = \text{Span}(\{v_1,\dots,v_n\})$. Any span is automatically a subspace, so no further checking is needed.

The span route is usually faster once you can rewrite the general element of $H$ as $a\,v_1 + b\,v_2 + \cdots$ for free parameters $a, b, \dots$

Example
H = {(a − 3b, b − a, a, b)} ⊂ R⁴
1
Write the general vector of $H$ as a linear combination in $a$ and $b$:
2
$\begin{pmatrix}a-3b\\ b-a\\ a\\ b\end{pmatrix} = a\begin{pmatrix}1\\-1\\1\\0\end{pmatrix} + b\begin{pmatrix}-3\\1\\0\\1\end{pmatrix}$
3
So $H = \text{Span}\big(\{(1,-1,1,0)^\top, (-3,1,0,1)^\top\}\big)$, automatically a subspace, with no axiom checking required.
03

Linear Combinations and Span

Concept
Span of a set of vectors

A linear combination of $v_1,\dots,v_n$ is a vector $\lambda_1 v_1 + \cdots + \lambda_n v_n$ for some scalars $\lambda_i$. The span collects every such combination:


$\text{Span}(\{v_1,\dots,v_n\}) = \{\lambda_1 v_1 + \cdots + \lambda_n v_n : \lambda_i \text{ any scalars}\}$

GeneratorsIf $H = \text{Span}(\{v_1,\dots,v_n\})$, we say $v_1,\dots,v_n$ generate $H$.
Interactive
Span of two vectors in R²
v1 blue · v2 pink · v1+v2 gold
2.01.0
1.02.0
Span(v1, v2) = R²
04

Linear Independence, Basis and Dimension

Concept
Linear independence

A set $\{v_1,\dots,v_n\}$ is linearly independent when the zero vector can only be built from them in the trivial way, using all zero coefficients. If some other combination also gives zero, the set is dependent.

$\lambda_1 v_1 + \cdots + \lambda_n v_n = 0 \implies \lambda_1 = \cdots = \lambda_n = 0$

Via rankArrange $v_1,\dots,v_n$ as columns of a matrix $A$: the vectors are independent exactly when $\text{rank}(A) = n$, that is, when $\det A \ne 0$ for a square matrix.
Interactive
Independent, or collapsed to a line?
v1 blue · v2 pink
2.01.0
-1.01.5
det = 4.00, linearly independent
Concept
Basis and dimension

A basis is a set of vectors that generates the whole space $V$, with no unnecessary vectors in it.


Every basis of the same $V$ has the same number of vectors. That number is called the dimension of $V$.


dim Rn = n dim Pn = n + 1
Example
Is {v1, v2, v3} a basis of R³?
1
Let $v_1=(3,0,-6)^\top$, $v_2=(-4,1,7)^\top$, $v_3=(-2,1,5)^\top$. Solve $xv_1+yv_2+zv_3=0$.
2
The coefficient matrix has rank 3, so the only solution is $x=y=z=0$: the set is linearly independent.
3
Because rank equals 3, the system $Ax=b$ also has a solution for every $b \in \mathbb{R}^3$: the set generates $\mathbb{R}^3$.
4
Independent and generating: $\{v_1,v_2,v_3\}$ is a basis of $\mathbb{R}^3$.
05

Fundamental Subspaces and the Rank Theorem

Concept
Fundamental subspaces of a matrix

Every matrix $A$ has three subspaces attached to it, built from its rows, its columns, and the solutions of $Ax=0$.

Nul(A)Col(A)Row(A)
Full nameNull spaceColumn spaceRow space
Defined as$\{x : Ax=0\}$Span of the columnsSpan of the rows
Subspace of$\mathbb{R}^n$$\mathbb{R}^m$$\mathbb{R}^n$
Dimensionfree variablespivot columnspivot rows
Read fromreduced echelon solutionoriginal matrix, pivot columnsany echelon form, nonzero rows

$\text{rank}\,A + \dim(\text{Nul}\,A) = n$, and $\dim(\text{Col}A) = \dim(\text{Row}A) = \text{rank}\,A$

For a square $n\times n$ matrix, $A$ is invertible if and only if $\text{Nul}A = \{0\}$.

Interactive
Rank, nullity and the Rank Theorem
[
]
n = 3
dim ColA = 2
dim NulA = 1
06

Coordinates and Change of Basis

Concept
Coordinates relative to a basis

Once you pick a basis $B = \{b_1,\dots,b_n\}$, every vector $v$ can be built from $b_1,\dots,b_n$ in exactly one way. Those weights are the coordinates of $v$ in that basis.


$[v]_B = (\lambda_1, \ldots, \lambda_n)$

In general, $[v]_X$ denotes the coordinates of $v$ relative to whichever basis $X$ is being used. If no basis is mentioned, we mean the standard basis $E$, where coordinates and components are the same thing: $[v]_E = v$.

Example
Finding [v]B for a custom basis
1
Let $B = \{b_1, b_2\}$ with $b_1 = (2,1)^\top$, $b_2 = (1,-1)^\top$, and let $v = (4,1)^\top$ in the standard basis, so $[v]_E = (4,1)^\top$.
2
Find $\lambda_1, \lambda_2$ solving $\lambda_1 b_1 + \lambda_2 b_2 = v$: $2\lambda_1 + \lambda_2 = 4$ and $\lambda_1 - \lambda_2 = 1$.
3
Solving gives $\lambda_1 = 5/3$, $\lambda_2 = 2/3$, so $[v]_B = (5/3,\, 2/3)^\top \approx (1.67,\, 0.67)^\top$.
Concept
Change of basis matrices

PE←B converts $B$-coordinates into standard coordinates. Its columns are the vectors of $B$ written in the standard basis: $P_{E\leftarrow B} = \begin{pmatrix}b_1 & \cdots & b_n\end{pmatrix}$, and $P_{E\leftarrow B}[v]_B = v$.


PB←E goes the other way, from standard coordinates to $B$-coordinates: $P_{B\leftarrow E} = (P_{E\leftarrow B})^{-1}$, and $P_{B\leftarrow E}\,v = [v]_B$.


PC←B converts $B$-coordinates directly into $C$-coordinates. Its columns are $[b_i]_C$: $P_{C\leftarrow B} = \begin{pmatrix}[b_1]_C & \cdots & [b_n]_C\end{pmatrix}$, and $P_{C\leftarrow B}[v]_B = [v]_C$.


Composing. $P_{D\leftarrow B} = P_{D\leftarrow C}\,P_{C\leftarrow B}$.

Diagram
Navigating between bases
B D C PD←B PC←B PD←C

Follow an arrow → multiply by its matrix. Reverse an arrow → invert it. For $B \rightarrow C \rightarrow D$, write the matrices right to left in the order you travel them: $P_{D\leftarrow B} = P_{D\leftarrow C}\,P_{C\leftarrow B}$.

$P_{C\leftarrow B}=\begin{pmatrix}1&1\\0&1\end{pmatrix},\; P_{D\leftarrow C}=\begin{pmatrix}2&0\\1&1\end{pmatrix} \;\Rightarrow\; P_{D\leftarrow B}=\begin{pmatrix}2&0\\1&1\end{pmatrix}\begin{pmatrix}1&1\\0&1\end{pmatrix}=\begin{pmatrix}2&2\\1&2\end{pmatrix}$
Example
From B coordinates to C coordinates
1
Let $b_1 = 4c_1+c_2$ and $b_2=-6c_1+c_2$, and let $x = 3b_1+b_2$, so $[x]_B = (3,1)^\top$.
2
The change of basis matrix is $P_{C\leftarrow B} = \begin{pmatrix}4 & -6\\ 1 & 1\end{pmatrix}$, built from the C coordinates of $b_1, b_2$.
3
$P_{C\leftarrow B}[x]_B = \begin{pmatrix}4 & -6\\ 1 & 1\end{pmatrix}\begin{pmatrix}3\\1\end{pmatrix} = \begin{pmatrix}6\\4\end{pmatrix} = [x]_C$
4
Since $P_{C\leftarrow B}$ is square and its columns are independent, it is invertible, and $P_{C\leftarrow B}^{-1} = P_{B\leftarrow C}$.