# Real Spectral Theorem

Aug-Nov 2020

## Recap

• Vector space $V$ over a scalar field $F$
• $F$: real field $\mathbb{R}$ or complex field $\mathbb{C}$ in this course
• $m\times n$ matrix A represents a linear map $T:F^n\to F^m$
• dim null $T+$ dim range $T=$ dim $V$
• Solution to $Ax=b$ (if it exists): $u+$ null$(A)$
• Four fundamental subspaces of a matrix
• Column space, row space, null space, left null space
• Eigenvalue $\lambda$ and Eigenvector $v$: $Tv=\lambda v$
• There is a basis w.r.t. which a linear map is upper-triangular
• If there is a basis of eigenvectors, linear map is diagonal w.r.t. it
• Inner products, norms, orthogonality and orthonormal basis
• There is an orthonormal basis w.r.t. which a linear map is upper-triangular
• Orthogonal projection: distance from a subspace
• Adjoint of a linear map: $\langle Tv,w\rangle=\langle v,T^*w\rangle$
• null $T=$ $($range $T^*)^{\perp}$
• Self-adjoint: $T=T^*$, Normal: $TT^*=T^*T$
• Eigenvectors corresponding to different eigenvalues are orthogonal
• Complex spectral theorem: $T$ is normal $\leftrightarrow$ orthonormal basis of eigenvectors

## Real vs complex vector spaces

Concept Real Complex
Linear maps - -
Fundamental spaces rowspace $=$ $($null$)^{\perp}$ Need not hold
Eigenvalues Need not be real At least one complex exists
Inner product Dot product Conjugate dot product

Example ($A$: $n\times n$ real matrix with real eigenvalue $\lambda$)

Then, eigenvector $\in$ null$(A-\lambda I)$ and is real

$A=\begin{bmatrix} 0&1\\ -1&0 \end{bmatrix}$

$\lambda = i,-i$

Eigenvectors: $(i,1)$, $(-i,1)$

$V$: vector space over $\mathbb{R}$

$T:V\to V$, self-adjoint

Eigenvalues of $T$ are real

$A$: $n\times n$ matrix representing $T$

Roots of det$(A-\lambda I)$ have to be all real.

## Real Spectral Theorem

$V$: vector space over $\mathbb{R}$

$T:V\to V$, an operator

The following are equivalent:

1. $T$ is self-adjoint
2. $V$ has an orthonormal basis of eigenvectors of $T$
3. $T$ is diagonal w.r.t. an orthonormal basis

Proof

Clearly, (2) implies (3) and vice versa

Proof of (3) implies (1)

Let $D$ be the diagonal matrix representing $T$ w.r.t. an orthonormal basis

Diagonal values of $D$ are the real eigenvalues

So, conjugate-transpose of $D$ is equal to $D$, or $T$: self-adjoint

## Proof of real spectral theorem (continued)

Proof of (1) implies (3)

Let $\lambda$ be a real eigenvalue of $T$ with a real eigenvector $v$.

Extend $v$ to an orthonormal basis: $\{v,u_1,\ldots,u_{n-1}\}$ (all real)

$A=\begin{bmatrix} \lambda&a_{12}&\cdots&a_{1n}\\ 0 & & & \\ \vdots & & A_1 & \\ 0 & & & \end{bmatrix}$ (matrix of $T$ w.r.t. above basis)

Since $A=A^T$, we have the following:

1. $a_{12}=\cdots=a_{1n}=0$

2. $A_1=A^T_1$: $(n-1)\times(n-1)$, self-adjoint

$A_1$: represents a self-adjoint operator from $\text{span}\{u_1,\ldots,u_{n-1}\}\to\text{span}\{u_1,\ldots,u_{n-1}\}$

Repeat same argument with $A_1$

Finally, get a diagonal matrix for $T$ w.r.t. an orthonormal basis for $V$

## Matrices of self-adjoint operators on real spaces

$A$: $n\times n$ real, symmetric matrix (representing a self-adjoint operator w.r.t. standard basis)

There is a real, orthonormal basis $\{e_1,\ldots,e_n\}$ s.t.

1. $e_i$ is an eigenvector of $A$, or $Ae_i=\lambda_i e_i$, $\lambda_i$ real

2. $A=\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_ne_n\overline{e^T_n}$

Example

$A=\begin{bmatrix} 14&-13&8\\ -13&14&8\\ 8&8&-7 \end{bmatrix}$

$e_1=\frac{1}{\sqrt{2}}(1,-1,0)$, $e_2=\frac{1}{\sqrt{3}}(1,1,1)$, $e_3=\frac{1}{\sqrt{6}}(1,1,-2)$

$\lambda_1=27$, $\lambda_2=9$, $\lambda_3=-15$

$A=\frac{27}{2}\begin{bmatrix}1\\-1\\0\end{bmatrix}\begin{bmatrix}1&-1&0\end{bmatrix} +\frac{9}{3}\begin{bmatrix}1\\1\\1\end{bmatrix}\begin{bmatrix}1&1&1\end{bmatrix} -\frac{15}{6}\begin{bmatrix}1\\1\\-2\end{bmatrix}\begin{bmatrix}1&1&-2\end{bmatrix}$

## Summary: Normal and self-adjoint operators

$\{e_1,\ldots,e_n\}$: orthonormal basis

$A=\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_ne_n\overline{e^T_n}$

Type $e_i$ $\lambda_i$
Normal (complex) complex complex
Powers of $A$
$A^k=\lambda^k_1e_1\overline{e^T_1}+\cdots+\lambda^k_ne_n\overline{e^T_n}$
Order eigenvalues by magnitude $\lvert\lambda_1\rvert \ge \cdots \ge \lvert\lambda_n\rvert$
$A^k\to\lambda^k_1e_1\overline{e^T_1}$ as $k\to\infty$ (assume $|\lambda_1|$ is the unique maximum)
Rank-$r$ approximation of $A$: $\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_re_r\overline{e^T_r}$