# Linear Functionals, Orthogonal Complements

Aug-Nov 2020

## Recap

• Vector space $V$ over a scalar field $F$
• $F$: real field $\mathbb{R}$ or complex field $\mathbb{C}$ in this course
• $m\times n$ matrix A represents a linear map $T:F^n\to F^m$
• dim null $T$ + dim range $T$ = dim $V$
• Linear equation: $Ax=b$
• Solution (if it exists): $u+$ null$(A)$
• Four fundamental subspaces of a matrix
• Column space, row space, null space, left null space
• Eigenvalue $\lambda$ and Eigenvector $v$: $Tv=\lambda v$
• Upper triangular matrix for a linear map
• Some linear maps are diagonalizable
• Inner products, norms
• Orthogonality and orthonormal basis
• Upper triangular matrix for a linear map over an orthonormal basis

## Linear functionals

$V$: vector space

A linear functional is a linear map from $V$ to $F$.

Examples

1. $\phi:\mathbb{R}^3\to\mathbb{R}$, $\phi(x_1,x_2,x_3)=2x_1-5x_2+7x_3$

2. $\phi:F^n\to F$, $\phi(x_1,\ldots,x_n)=c_1x_1+\cdots+c_nx_n$

All linear functionals from $F^n$ to $F$ will be like above

3. $P_2(\mathbb{R})$: polynomials of deg $\le 2$, $\phi:P_2(\mathbb{R})\to\mathbb{R}$ defined as $\phi(p(x))=\int_0^1 p(x)\cos(\pi x)dx$

• Is there a simpler description?

## Linear functionals in inner product spaces

$V$: inner product space

Example of linear functional: Fix $u\in V$. $\phi(v)=\langle v,u\rangle$

Can there be any other type?

Riesz representation theorem: $V$ is a finite-dimensional inner product space and $\phi$ is a linear functional on $V$. Then, there exists a unique vector $u$ such that $\phi(v)=\langle v,u\rangle$.

Proof

$e_1,\cdots,e_n$: orthonormal basis for $V$

\begin{align} \phi(v)&=\phi(\langle v,e_1\rangle e_1+\cdots+\langle v,e_n\rangle e_n)\\ &=\langle v,e_1\rangle \phi(e_1) + \cdots + \langle v,e_n\rangle \phi(e_n)\\ &=\langle v,\overline{\phi(e_1)}e_1+\cdots+\overline{\phi(e_n)}e_n\rangle. \end{align}

So, $u=\overline{\phi(e_1)}e_1+\cdots+\overline{\phi(e_n)}e_n$ is such that $\phi(v)=\langle v,u\rangle$.

## Example

$P_2(\mathbb{R})$: polynomials of deg $\le 2$, Inner product: $\langle p,q\rangle=\int\limits_0^1 p(x)q(x)dx$

Linear functional $\phi:P_2(\mathbb{R})\to\mathbb{R}$ defined as $\phi(p(x))=\int_0^1 p(x)\cos(\pi x)dx$

Riesz: There exists $q(x)=q_0+q_1x+q_2x^2$ such that $\phi(p(x))=\int_0^1 p(x)\cos(\pi x)dx=\int_0^1 p(x)q(x)dx$

1. Find orthonormal basis $e_0(x),e_1(x),e_2(x)$

• Gram-Schmidt on $1,x,x^2$.
2. $q(x)=\phi(e_0(x))e_0(x)+\phi(e_1(x))e_1(x)+\phi(e_2(x))e_2(x)$

## Orthogonal complements

$U\subseteq V$, subset

Orthogonal complement of $U$, denoted $U^{\perp}$, is the set of vectors orthogonal to every vector in $U$. $U^{\perp}=\{v\in V: \langle v,u\rangle=0\text{ for every }u\in U\}$

Examples: $\mathbb{R}^2$

• $U=0$

• $U=(x_1,y_1)$

• $U$: line through origin

• $U=\{(x_1,y_1),(x_2,y_2)\}$

• $U=V$

## Basic properties: $U$ is a subset of $V$

1. $U^{\perp}$ is a subspace of $V$

2. $\{0\}^{\perp}=V$, $V^{\perp}=\{0\}$

3. $U\cap U^{\perp}=\emptyset$ or $\{0\}$

4. if $U\subseteq W$, $W^{\perp}\subseteq U^{\perp}$

## Examples: $U$ is a subspace of $\mathbb{R}^n$ or $\mathbb{C}^n$

$U=\text{span}\{(1,2,3,4),(3,4,5,6)\}$

$U=$ rowspace$(A)$, where $A=\begin{bmatrix} 1&2&3&4\\ 3&4&5&6 \end{bmatrix}$

\begin{align} U^{\perp}&=\{v\in \mathbb{R}^4:\langle v,(1,2,3,4) \rangle=0, \langle v,(3,4,5,6) \rangle=0\}\\ &=\{v\in\mathbb{R}^4:Av=0\} \end{align}

$U^{\perp}=$ null$(A)$

=======================================================

$U$ is a subspace of $\mathbb{C}^n$, $U=$ rowspace$(A)$

$U^{\perp}=$ null$(\overline{A})$, $\overline{A}$: element-wise conjugate of $A$

## Four fundamental subspaces of $m\times n$ real matrix

$\langle x,y \rangle=x_1y_1+\cdots+x_ny_n$

$A$: $m\times n$ matrix

Result 1

null$(A)=$ rowspace$(A)^{\perp}$

Result 2

range$(A)=$ left-null$(A)^{\perp}$

## Orthogonal complement of subspaces

if $U$: finite-dimensional subspace of $V$, then $V=U\oplus U^{\perp}$

Proof

$e_1,\ldots,e_m$: orthonormal basis for $U$

Any $v\in V$ can be written as $v=(\langle v,e_1\rangle e_1+\cdots+\langle v,e_m\rangle e_m)+(v-\langle v,e_1\rangle e_1-\cdots-\langle v,e_m\rangle e_m)$ First term is in $U$, and second term in $U^{\perp}$ (why?)

Since $U \cap U^{\perp}=\{0\}$, $V=U\oplus U^{\perp}$.

Corollary ($V$: finite-dimensional)

dim $U^{\perp}=$ dim $V-$ dim $U$

## $U$: subspace of a finite-dimensional $V$

Complement of complement

$\left(U^{\perp}\right)^{\perp}=U$

Orthonormal basis for $U^{\perp}$

1. Find an orthonormal basis for $U$: $e_1,\ldots,e_m$

2. Extend to orthonormal basis of $V$: $e_1,\ldots,e_m,e_{m+1},\ldots,e_n$

3. Orthonormal basis of $U^{\perp}$: $e_{m+1},\ldots,e_n$

To specify $U$

Give a basis for $U$

or

Give a basis for $U^{\perp}$

## Sum and intersection of subspaces

$U$: basis $u_1,\ldots,u_k$, $W$: basis $w_1,\ldots,w_l$

sum of $U$ and $W$

$U+W$: spanning set $u_1,\ldots,u_k,w_1,\ldots,w_l$

Reduce by elementary row operations

intersection of $U$ and $W$

$(U+W)^{\perp}=U^{\perp}\cap W^{\perp}$

Proof

If $v\in(U+W)^{\perp}$, $v\in U^{\perp}$ and $v\in W^{\perp}$

$U^{\perp}=\{v\in V: \langle v,u_i \rangle=0, i=1,\ldots,k\}$

$W^{\perp}=\{v\in V: \langle v,w_j \rangle=0, j=1,\ldots,l\}$

$U^{\perp}\cap W^{\perp}=\{v\in V: \langle v,u_i \rangle=0, i=1,\ldots,k\text{ and }\langle v,w_j \rangle=0, j=1,\ldots,l\}$

Corollary: $U\cap W=(U^{\perp}+W^{\perp})^{\perp}$

## Example

$U=$ rowspace$\begin{bmatrix} 1&2&3&4\\ 3&4&5&6 \end{bmatrix}$, $W=$ rowspace$\begin{bmatrix} 1&1&1&1\\ 1&-1&1&-1 \end{bmatrix}$

$U=$ null$\begin{bmatrix} 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}$, $W=$ null$\begin{bmatrix} -1&0&1&0\\ 0&-1&0&1 \end{bmatrix}$

$U\cap W=$ null$\begin{bmatrix} 1&0&-1&0\\ 0&1&0&-1\\ 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}=$ null$\begin{bmatrix} 1&0&0&-1\\ 0&1&0&-1\\ 0&0&1&-1 \end{bmatrix}$

$U\cap W=$ rowspace$\begin{bmatrix}1&1&1&1\end{bmatrix}$