# Projection and distance from a subspace

Aug-Nov 2020

## Recap

• Vector space $V$ over a scalar field $F$
• $F$: real field $\mathbb{R}$ or complex field $\mathbb{C}$ in this course
• $m\times n$ matrix A represents a linear map $T:F^n\to F^m$
• dim null $T$ + dim range $T$ = dim $V$
• Solution to $Ax=b$ (if it exists): $u+$ null$(A)$
• Four fundamental subspaces of a matrix
• Column space, row space, null space, left null space
• Eigenvalue $\lambda$ and Eigenvector $v$: $Tv=\lambda v$
• Some linear maps are diagonalizable
• Inner products, norms, orthogonality and orthonormal basis
• Upper triangular matrix for a linear map over an orthonormal basis
• $V=U\oplus U^{\perp}$ for any subspace $U$
• Orthogonal projection

## Distances and norms

$v_1,v_2\in V$

Distance between $v_1$ and $v_2$, $d(v_1,v_2)$, is defined as $d(v_1,v_2)=\lVert v_1-v_2\rVert$

Properties

• $d(v_1,v_2)=0$ iff $v_1=v_2$

• $d(v_1,v_2)^2=\langle v_1-v_2,v_1-v_2\rangle$

How to define distance between a vector and a subspace?

## Distance of a vector from a subspace

$v\in V$ and $U$: subspace

Distance between $v$ and $U$, $d(v,U)$, is defined as $d(v,U)=\min_{u\in U}\lVert v-u\rVert$

Properties

• $d(v,U)=0$ iff $v\in U$

• $Tx=v$ has a solution iff $d(v,\text{range } T)=0$

Examples

1. $\mathbb{R}^2$: $d((x,y),x\text{-axis})=?$

2. $\mathbb{R}^2$: $d((2,5),\{(x,y):x=y\})=?$

3. $\mathbb{R}^2$: $d((2,5),\{(x,y):ax+by=0\})=?$

## Closest point in subspace

$v\in V$ and $U$: subspace

Point closest to $v$ in $U$ is defined as $\arg\min_{u\in U}\lVert v-u\rVert$

• How is “closest” justified? Is it unique?

• Distance of $v$ from $U$ is the distance between $v$

and the point closest to $v$ in $U$

Examples

1. $\mathbb{R}^3$: $(1,2,3)$ and $x{-}y$ plane?

2. $\mathbb{R}^3$: $(1,2,3)$ and $\{(x,y,z):x+y+z=x+2y+3z=0\}$?

3. $\mathbb{R}^{100}$: $(1,2,\ldots,100)$ and a 50-dimensional subspace?

## Orthogonal projection and minimization

$v\in V$ and $U$: subspace

$P_U$: orthogonal projection onto $U$

Closest point to $v$ in $U$ is $P_Uv$ (unique).

Distance of $v$ from $U$ is $\lVert v-P_Uv\rVert$.

Proof

Let $u\in U$ be some vector in $U$.

\begin{align} \lVert v-u\rVert^2&=\lVert (v-P_Uv)+(P_Uv-u)\rVert^2\\ &=\lVert v-P_Uv\rVert^2+\lVert P_Uv-u\rVert^2\\ &\ge \lVert v-P_Uv\rVert^2 \end{align}

## Examples: $\mathbb{R}^2$

1. $(x,y)$ and $x$-axis

$U=$ span$\{(1,0)\}$

$P_U(x,y)=\langle (x,y),(1,0)\rangle (1,0)=(x,0)$

2. $(x,y)$ and $\{(x,y):x=y\}$

$U=$ span$\{(1/\sqrt{2},1/\sqrt{2})\}$

$P_U(x,y)=((x+y)/2,(x+y)/2)$

3. $(x,y)$ and $\{(x,y):ax+by=0\}$

$U=$ span$\{(b,-a)\}=$ span$\{(b/\sqrt{a^2+b^2},-a/\sqrt{a^2+b^2})\}$

$P_U(x,y)=((bx-ay)b/(a^2+b^2),-(bx-ay)a/(a^2+b^2))$

## Examples

1. $(x,y,z)$ and $x{-}y$ plane

$U=$ span$\{(1,0,0),(0,1,0)\}$

$P_U(x,y,z)=x(1,0,0)+y(0,1,0)=(x,y,0)$

2. $(x,y,z)$ and $\{(x,y,z):x+y+z=x+2y+3z=0\}$

$U=$ span$\{(1,-2,1)/\sqrt{6}\}$

$P_U(x,y,z)=(x-2y+z)(1,-2,1)/6$

3. $(1,2,\ldots,100)$ and a 50-dimensional subspace?