Linear Algebra

Linear Transformations

↩ Back
01

Transformations, Domain and Codomain

Concept
Transformation

A transformation $T$ from $U$ to $V$ assigns to each vector $x \in U$ a vector $T(x) \in V$, called the image of $x$ under $T$.

$T : U \longrightarrow V, \quad x \longmapsto T(x)$

Domain$U$, the set every input is drawn from.
Codomain$V$, the set $T$ maps into.
Image space$\operatorname{Im}T = \{T(x) : x \in U\} \subset V$.
Concept
Injective, surjective, bijective

For $T: U \to V$, comparing solutions of $T(x) = b$ across every $b \in V$ tells us which of these $T$ is.

Injective$T(x) = b$ has at most one solution, for every $b \in V$.
Surjective$T(x) = b$ has at least one solution, for every $b \in V$.
Bijective$T(x) = b$ has exactly one solution, for every $b \in V$.

Injective, not surjective. $T:\mathbb{R}\to\mathbb{R}^2$, $x \mapsto (x,0)^\top$. No element has image $(0,1)^\top$.
Surjective, not injective. $T:\mathbb{R}^2\to\mathbb{R}$, $(x,y)^\top \mapsto x$. Both $(0,0)^\top$ and $(0,1)^\top$ map to $0$.
Example
Domain, codomain and image space
Let $T:\mathbb{R}^2\to\mathbb{R}^3$, $(x,y)^\top \mapsto (x,\,y,\,x+y)^\top$.
The domain is $\mathbb{R}^2$, and the codomain is $\mathbb{R}^3$.
The image space is $\operatorname{Im}T = \{(x,y,x+y)^\top : x,y \in \mathbb{R}\} = \{(a,b,c)^\top \in \mathbb{R}^3 : c = a+b\}$, a plane through the origin.
This plane is a proper subset of $\mathbb{R}^3$, so $T$ is not surjective, even though it is injective.
02

Linear vs Non-linear Transformations

Example
A transformation that is not linear
Consider $T:\mathbb{R}\to\mathbb{R}$, $x \mapsto x^2$. A single counterexample is enough to break linearity.
Take $x=1$: $T(1+1) = T(2) = 2^2 = 4$, while $T(1) + T(1) = 1^2 + 1^2 = 2$.
Since $4 \neq 2$, additivity fails: $T(1+1) \neq T(1)+T(1)$.
One failure of either property is enough, so $T$ is not linear. No further checking is needed.
Example
A transformation that is linear

Consider $T:\mathbb{R}^3\to\mathbb{R}^2$, $(x,y,z)^\top \mapsto (2x-y+z,\ 2x-z)^\top$.

Take $u=(1,2,-1)^\top$ and $v=(0,3,2)^\top$, so $u+v=(1,5,1)^\top$.
$T(u)=(-1,3)^\top$ and $T(v)=(-1,-2)^\top$, so $T(u)+T(v)=(-2,1)^\top$.
$T(u+v)=(-2,1)^\top$ as well: it matches $T(u)+T(v)$.
For $\lambda=3$: $T(\lambda u)=T(3,6,-3)^\top=(-3,9)^\top$, equal to $\lambda T(u)=3(-1,3)^\top=(-3,9)^\top$.

T behaves this way for any vectors and any scalar, not just this pair, so T is linear. Section 03 shows why this always holds, once T is written as matrix multiplication.
03

Matrix Representation

Theorem
T is linear ⇔ T has a matrix representation
$T:\mathbb{R}^n \to \mathbb{R}^m$ is linear $\quad\Longleftrightarrow\quad$ there exists a matrix $A$ with $T(x) = Ax$ for all $x \in \mathbb{R}^n$
Example
Building the standard matrix column by column

Recall $T:\mathbb{R}^3\to\mathbb{R}^2$, $(x,y,z)^\top \mapsto (2x-y+z,\ 2x-z)^\top$, shown to be linear in Section 02.

$T(e_1) = T(1,0,0)^\top = (2,\,2)^\top$, the first column of $A$.
$T(e_2) = T(0,1,0)^\top = (-1,\,0)^\top$, the second column of $A$.
$T(e_3) = T(0,0,1)^\top = (1,\,-1)^\top$, the third column of $A$.

$A = \begin{pmatrix} 2 & -1 & 1 \\ 2 & 0 & -1 \end{pmatrix}, \qquad Ax = \begin{pmatrix} 2 & -1 & 1 \\ 2 & 0 & -1 \end{pmatrix}\begin{pmatrix} x\\y\\z \end{pmatrix} = \begin{pmatrix} 2x-y+z \\ 2x-z \end{pmatrix} = T(x,y,z)$

Column $i$ of $A$ is the image of the $i$-th standard basis vector $e_i$. Finding $x$ such that $T(x) = b$ is exactly solving the linear system $Ax = b$.

04

Kernel, Cokernel and Image Space

Concept
Fundamental subspaces of T

Let $T: \mathbb{R}^n \to \mathbb{R}^m$ be linear with standard matrix $A$.

ker T$\{x : Ax = 0\}$, the kernel of $T$.
coker T$\{x : A^\top x = 0\}$, the cokernel of $T$.
Im T$\{T(x) : x \in \mathbb{R}^n\}$, the image space of $T$.
Proposition
Relation with the fundamental subspaces of A
Subspace of TEquals
ker Tthe null space of $A$, Nul $A$
Im Tthe column space of $A$, Col $A$
coker Tthe null space of $A^\top$, Nul $A^\top$

So every question about $T$ (whether it is injective, surjective, what it maps onto) reduces to a question about the four fundamental subspaces of its standard matrix $A$.

05

Injective, Surjective and Bijective via Rank

Theorem
Equivalent characterizations

Let $T:\mathbb{R}^n \to \mathbb{R}^m$ be linear with standard matrix $A$, of size $m \times n$.

Bijective = Injective AND Surjective
T is injective whenT is surjective whenT is bijective when
ker T = {0}ker T = {0}
Im T = $\mathbb{R}^m$Im T = $\mathbb{R}^m$
coker T = {0}coker T = {0}
columns of A are linearly independentrows of A are linearly independentcolumns and rows of A are linearly independent
rank(A) = nrank(A) = mrank(A) = n = m
Ax = b has at most one solution, for every bAx = b has at least one solution, for every bAx = b has exactly one solution, for every b
Corollary
What the shape of A already tells you
More columns than rowsT can never be injective.
More rows than columnsT can never be surjective.
Interactive
A 3×2 matrix

A is $3\times2$ (more rows than columns), so $T:\mathbb{R}^2 \to \mathbb{R}^3$. Rank decides whether T is injective; it can never be surjective here.

[
]
rank(A) = 2
Interactive
A 2×3 matrix

A is $2\times3$ (more columns than rows), so $T:\mathbb{R}^3 \to \mathbb{R}^2$. Rank decides whether T is surjective; it can never be injective here.

[
]
rank(A) = 2
06

Change of Basis

Concept
The matrix ABC

Let $U$ have basis $B=\{b_1,\dots,b_n\}$ and $V$ have basis $C=\{c_1,\dots,c_m\}$, and let $T: U \to V$ be linear.

$A_{BC} = \begin{pmatrix} [T(b_1)]_C & \cdots & [T(b_n)]_C \end{pmatrix}$

$A_{BC}$ turns the $B$-coordinates of any $v \in U$ into the $C$-coordinates of $T(v)$:

$[T(v)]_C = A_{BC}[v]_B$

When $U=\mathbb{R}^n$, $V=\mathbb{R}^m$ and both use the standard basis, $A_{BC}$ is exactly the standard matrix $A$.


Every $P_{X\leftarrow Y}$ here denotes a change of basis matrix, converting $Y$-coordinates into $X$-coordinates. For more on how these matrices are built and used, see the Vector Spaces visual guide.

Diagram
Standard basis vs custom basis
E₃ E₂ B C A ABC PE₃←B PC←E₂

Follow the diagram: go up from B to E₃ (multiply by $P_{E_3\leftarrow B}$), across with $A$, then down from E₂ to C (multiply by $P_{C\leftarrow E_2}$). In the formula below, that same order reads right to left.

$A_{BC}$ = $P_{C\leftarrow E_2}$ $A$ $P_{E_3\leftarrow B}$
Example
Computing ABC

The transformation. Recall $T:\mathbb{R}^3\to\mathbb{R}^2$, $(x,y,z)^\top \mapsto (2x-y+z,\ 2x-z)^\top$, with standard matrix $A = \begin{pmatrix}2&-1&1\\2&0&-1\end{pmatrix}$.

Basis of $\mathbb{R}^3$. $B=\{b_1,b_2,b_3\}$, with $b_1=(1,1,-1)^\top$, $b_2=(0,1,1)^\top$, $b_3=(1,0,2)^\top$.

Basis of $\mathbb{R}^2$. $C=\{c_1,c_2\}$, with $c_1=(2,0)^\top$, $c_2=(1,-1)^\top$.

$P_{E_3\leftarrow B}$. Built from the columns of $B$ directly: $P_{E_3\leftarrow B} = \begin{pmatrix}1&0&1\\1&1&0\\-1&1&2\end{pmatrix}$.

$P_{C\leftarrow E_2}$. The inverse of $P_{E_2\leftarrow C} = \begin{pmatrix}2&1\\0&-1\end{pmatrix}$, giving $P_{C\leftarrow E_2} = \frac{1}{2}\begin{pmatrix}1&1\\0&-2\end{pmatrix}$.

Result. $A_{BC} = P_{C\leftarrow E_2}\, A\, P_{E_3\leftarrow B} = \frac{1}{2}\begin{pmatrix}3&-1&4\\-6&2&0\end{pmatrix}$.


07

Composition of Linear Transformations

Concept
Composing two linear maps

If $T: U \to V$ and $S: V \to W$ are both linear, chaining them gives a new map $U \to W$.

$(S\circ T)(u) = S(T(u))$

$S\circ T$ is linear too. If $A$ is the standard matrix of $T$ and $B$ is the standard matrix of $S$, the standard matrix of $S\circ T$ is just the product $BA$.


Order matters: in $S\circ T$, $T$ acts first (it is closest to $u$), then $S$ acts on the result. That is why $A$ appears on the right in $BA$.

Diagram
Three different vector spaces
P₂ T S S ∘ T

The straight arrows go step by step, $T$ then $S$. The curved arrow underneath is the shortcut $S\circ T$, doing both steps at once, first $T$ then $S$, read right to left in the name.

Example
R³ → R² → polynomials
Given $T:\mathbb{R}^3\to\mathbb{R}^2$ and $S:\mathbb{R}^2\to P_2$ below, find $S\circ T:\mathbb{R}^3\to P_2$, the map that sends a vector directly to a polynomial.

Given

$T$. $T:\mathbb{R}^3\to\mathbb{R}^2$, $(x,y,z)^\top \mapsto (x+y,\ y+z)^\top$.

$S$. $S:\mathbb{R}^2\to P_2$, $(a,b)^\top \mapsto a + bt + (a-b)t^2$.


Solving it as a mapping

Apply $T$ first. $T(x,y,z) = (x+y,\ y+z)$.

Then apply $S$. $(S\circ T)(x,y,z) = (x+y) + (y+z)t + \bigl((x+y)-(y+z)\bigr)t^2$.

Simplify. $(x+y)-(y+z) = x - z$, so $(S\circ T)(x,y,z) = (x+y) + (y+z)t + (x-z)t^2$.


Solving it with matrices

Matrix of $T$. Columns $T(e_1),T(e_2),T(e_3)$: $A = \begin{pmatrix}1&1&0\\0&1&1\end{pmatrix}$.

Matrix of $S$. With respect to the basis $\{1,t,t^2\}$ of $P_2$, columns $S(1,0)$ and $S(0,1)$: $B = \begin{pmatrix}1&0\\0&1\\1&-1\end{pmatrix}$.

Product. The matrix of $S\circ T$ is $BA$, with $A$ on the right since $T$ acts first: $BA = \begin{pmatrix}1&0\\0&1\\1&-1\end{pmatrix}\begin{pmatrix}1&1&0\\0&1&1\end{pmatrix} = \begin{pmatrix}1&1&0\\0&1&1\\1&0&-1\end{pmatrix}$.


Check

Reading off the rows of $BA$ against $(x,y,z)^\top$ gives back the same coefficients as the mapping approach: $x+y$, $y+z$, $x-z$.