Diagonalization

Definition

Eigenvalue and eigenvector

Let $A \in M_{n\times n}$. A vector $x$ is an eigenvector of $A$ when $Ax = \lambda x$ for some scalar $\lambda$. That scalar $\lambda$ is the eigenvalue associated with $x$.

$Ax = \lambda x, \qquad x \neq 0$

Definition

Eigenspace

Let $A \in M_{n\times n}$. The eigenspace of $\lambda$ is the subspace $\operatorname{Nul}(A-\lambda I)$.

$E_\lambda = \operatorname{Nul}(A-\lambda I)$

Nontrivial kernel$\lambda$ is an eigenvalue when $(A-\lambda I)x=0$ has solutions besides $x=0$

Example

Checking that 7 is an eigenvalue

Let $A=\begin{pmatrix}1&6\\5&2\end{pmatrix}$. To check that $7$ is an eigenvalue we look for nontrivial solutions of $(A-7I)x=0$.

$A-7I = \begin{pmatrix}1&6\\5&2\end{pmatrix} - \begin{pmatrix}7&0\\0&7\end{pmatrix} = \begin{pmatrix}-6&6\\5&-5\end{pmatrix}$, whose columns are dependent, so $7$ is an eigenvalue.

Row reducing gives $\begin{pmatrix}1&-1\\0&0\end{pmatrix}$, so the solutions are $x=y$, that is, $E_7 = \operatorname{Span}\left\{\begin{pmatrix}1\\1\end{pmatrix}\right\}$.

Interactive

Is v an eigenvector of A?

Rotate $v$ (blue) and change its length. $Av$ (red) is drawn alongside it. When both arrows line up, $v$ sits on an invariant line and is an eigenvector, with $\lambda \approx |Av|/|v|$ (negative when they point opposite ways).

R²

Invariant lines

Controls

Matrix A and vector v

(

)

Angle θ π/4

|v| 2.5

Av = (0, 0)

Definition

Characteristic equation and polynomial

For $A \in M_{n\times n}$, $\det(A-\lambda I)$ is a polynomial in $\lambda$ of degree $n$: the characteristic polynomial of $A$. Setting it to 0 gives the characteristic equation, and its roots are the eigenvalues of $A$.

$\det(A-\lambda I) = 0$

Example

A triangular matrix

Let $A=\begin{pmatrix}5&-2&6&-1\\0&3&-8&0\\0&0&5&4\\0&0&0&1\end{pmatrix}$. Since $A-\lambda I$ is triangular, its determinant is the product of the diagonal entries.

$\det(A-\lambda I) = (5-\lambda)(3-\lambda)(5-\lambda)(1-\lambda) = (5-\lambda)^2(3-\lambda)(1-\lambda)$.

The roots are $\lambda=5,3,1$. So $5$ is an eigenvalue of multiplicity 2, while $3$ and $1$ have multiplicity 1.

Triangular shortcutfor a triangular matrix the eigenvalues are simply the diagonal entries

Theorem

Independence and diagonalizability conditions

If $v_1,\dots,v_r$ are eigenvectors of $A$ corresponding to different eigenvalues $\lambda_1,\dots,\lambda_r$, then $\{v_1,\dots,v_r\}$ is linearly independent. In particular, $A\in M_{n\times n}$ has at most $n$ eigenvalues.

Let $\lambda_1,\dots,\lambda_p$ be the different eigenvalues of $A$.

$\dim E_{\lambda_k} \le \text{multiplicity of } \lambda_k$

$A$ is diagonalizable if and only if these dimensions add up to $n$.

Equivalently, $A$ is diagonalizable when it has $n$ linearly independent eigenvectors.

$A = PDP^{-1}$

Columns of Pthe $n$ independent eigenvectors

Diagonal of Dtheir eigenvalues, in that same order

Method

Steps to diagonalize a matrix

Find the eigenvalues of $A$: compute the roots of the characteristic polynomial.

For each eigenvalue $\lambda$, compute the kernel of $A-\lambda I$ to get its eigenvectors. If together they give $n$ independent eigenvectors, continue; otherwise $A$ is not diagonalizable.

Build $P$, whose columns are the $n$ eigenvectors found in step 2.

Build $D$, the diagonal matrix of the matching eigenvalues, listed in the same order as the columns of $P$, each repeated as many times as it has independent eigenvectors.

To verify the result without inverting $P$, it is enough to check $AP = PD$.

Example

Diagonalizing a 3×3 matrix

Let $A=\begin{pmatrix}1&3&3\\-3&-5&-3\\3&3&1\end{pmatrix}$. The characteristic equation is $-(\lambda-1)(\lambda+2)^2=0$, so the eigenvalues are $1$ and $-2$, the latter with multiplicity 2.

For $\lambda=1$: the kernel of $A-I$ gives $E_1 = \operatorname{Span}\left\{(1,-1,1)^\top\right\}$.

For $\lambda=-2$: the kernel of $A+2I$ gives the plane $x+y+z=0$, so $E_{-2} = \operatorname{Span}\left\{(1,0,-1)^\top,(1,-1,0)^\top\right\}$, dimension 2, matching the multiplicity.

Three independent eigenvectors in total, so $A$ is diagonalizable: $P=\begin{pmatrix}1&1&1\\-1&0&-1\\1&-1&0\end{pmatrix}$, $D=\begin{pmatrix}1&0&0\\0&-2&0\\0&0&-2\end{pmatrix}$.

Proposition

Powers of a diagonalizable matrix

If $A=PDP^{-1}$, then $A^n=PD^nP^{-1}$, since $D$ is diagonal and $D^n$ is just the diagonal matrix of the $\lambda_k^n$.

$A^n = PD^nP^{-1}$

Example

Reusing a diagonalization

Take $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$, with eigenvalues $3$ and $1$, eigenvectors $(1,1)^\top$ and $(1,-1)^\top$.

$P=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$, $D=\begin{pmatrix}3&0\\0&1\end{pmatrix}$, $P^{-1}=\dfrac12\begin{pmatrix}1&1\\1&-1\end{pmatrix}$.

$D^{10}=\begin{pmatrix}3^{10}&0\\0&1\end{pmatrix}$, so $A^{10}=PD^{10}P^{-1}$ needs only two powers of scalars, not nine matrix products.

Theorem

Symmetric ⇔ orthogonally diagonalizable

$A$ is symmetric when $A=A^\top$. $A$ is orthogonally diagonalizable when it is symmetric.

$A$ is orthogonally diagonalizable when $A=PDP^\top$ for some diagonal $D$ and some $P$ with orthonormal columns, that is, $P^{-1}=P^\top$.

$A = PDP^\top = PDP^{-1}$

Eigenvectors from different eigenspaces of a symmetric matrix are orthogonal to each other, so the whole basis of eigenvectors can be chosen orthonormal.

Summary

Key results by concept

Concept	What it gives you
Eigenvalue / eigenvector	$Ax=\lambda x$
Characteristic equation	$\det(A-\lambda I)=0$, its solutions are the eigenvalues
Independence	eigenvectors of different eigenvalues are linearly independent
Diagonalizability	$A=PDP^{-1}$ possible iff eigenspace dimensions add up to $n$
High powers	$A^n=PD^nP^{-1}$, turning matrix products into scalar powers
Symmetric matrices	always orthogonally diagonalizable, $A=PDP^\top$

Eigenvalues and Eigenvectors

The Characteristic Equation

Independence and Diagonalizability Conditions

Diagonalization

Application: High Powers of a Matrix

Diagonalization of Symmetric Matrices