Aug-Nov 2020

Recap

• Vector space $V$ over a scalar field $F$
• $F$: real field $\mathbb{R}$ or complex field $\mathbb{C}$ in this course
• $m\times n$ matrix A represents a linear map $T:F^n\to F^m$
• dim null $T$ + dim range $T$ = dim $V$
• Solution to $Ax=b$ (if it exists): $u+$ null$(A)$
• Four fundamental subspaces of a matrix
• Column space, row space, null space, left null space
• Eigenvalue $\lambda$ and Eigenvector $v$: $Tv=\lambda v$
• If there is a basis of eigenvectors, linear map is diagonal w.r.t. it
• Inner products, norms, orthogonality and orthonormal basis
• There is an orthonormal basis w.r.t. which a linear map is upper-triangular
• Orthogonal projection: distance from a subspace
• Adjoint of a linear map: $\langle Tv,w\rangle=\langle v,T^*w\rangle$
• null $T=$ $($range $T^*)^{\perp}$

Eigenvalues of $ST$ and $TS$

$S:V\to W$, $T:W\to V$

$ST:W\to W$ and $TS: V\to V$ are operators

If $\lambda\ne0$ is an eigenvalue of $ST$, then it is an eigenvalue of $TS$ as well.

Proof

$v$: $STv=\lambda v$, $\lambda\ne0$

Then, $Tv\ne0$,

and $\lambda Tv=T(\lambda v)=T(STv)=(TS)Tv$

So, $Tv$: eigenvector of $TS$ with eigenvalue $\lambda$

More on $AB$ and $BA$

$A$: $m\times n$ and $B$: $n\times m$

$A$, $B$: represent suitable linear maps

$AB$: $m\times m$, $BA$: $n\times n$

$t^n\text{det}(tI_m-AB)=t^m\text{det}(tI_n-BA)$

Proof

$C=\begin{bmatrix} tI_m&A\\ B&I_n \end{bmatrix}$, $D=\begin{bmatrix} I_m&0\\ -B&tI_n \end{bmatrix}$

det$(CD)=$ det$(DC)$

det$\begin{bmatrix} tI_m-AB&tA\\ 0&tI_n \end{bmatrix}=$ det$\begin{bmatrix} tI_m&A\\ 0&tI_n-BA \end{bmatrix}$

$n>m$

$\{$Eigenvalues of $BA\}=\{$Eigenvalues of $AB\}\cup(n-m)$ zeros

$V$: finite-dimensional inner product space

$T:V\to V$, an operator

$T^*:V\to V$, adjoint of $T$

Properties

• $\langle Tu,v\rangle=\langle u,T^*v\rangle$

• null $T^*=$ $($range $T)^{\perp}$

• dim range $T=$ dim range $T^*$

• dim null $T=$ dim null $T^*$

Eigenvalues

$T:V\to V$, an operator

if $\lambda$ is an eigenvalue of $T$, $\bar{\lambda}$ is an eigenvalue of $T^*$

Proof

dim range $(T-\lambda I)<$ dim $V$

$(T-\lambda I)^* = T^*-\bar{\lambda}I$

Since dim range $(T-\lambda I)=$ dim range $(T^*-\bar{\lambda}I)$,

dim range $(T^*-\bar{\lambda}I)<$ dim $V$

Eigenvalues of $TT^*$ and $T^*T$

$T:V\to W$, $T^*:W\to V$

$TT^*: W\to W$, $T^*T: V\to V$

Nonzero eigenvalues of $TT^*$ and $T^*T$ are the same

Differences only in additional zero eigenvalues for one of the two

Let dim $V\ge$ dim $W$

Singular values of a linear map $T$: eigenvalues of $TT^*$

$T:V\to W$, linear map and $w\in W$

Least squares: Solve $\min\limits_v\,\lVert Tv-w\rVert$

Solution: $Tv-w$ is orthogonal to range $T$, or

$Tv-w\in$ $($range $T)^{\perp}$

Since null $T^*=$ $($range $T)^{\perp}$, we require

$T^*(Tv-w)=0$

Normal equation: $T^*Tv=T^*w$

Suppose $T^*T$ is invertible

Pseudo-inverse of $T$: $(T^* T)^{-1}T^*$

Why? $((T^* T)^{-1}T^*)T=I$