# Dot product and length in $\mathbb{C}^n$, Inner product and norm in $V$ over $F$

Aug-Nov 2020

## Recap

• Vector space $V$ over a scalar field $F$
• $F$: real field $\mathbb{R}$ or complex field $\mathbb{C}$ in this course
• $m\times n$ matrix A represents a linear map $T:F^n\to F^m$
• dim null $T$ + dim range $T$ = dim $V$
• Linear equation: $Ax=b$
• Solution (if it exists): $u+$ null$(A)$
• Four fundamental subspaces of a matrix
• Column space, row space, null space, left null space
• Eigenvalue $\lambda$ and Eigenvector $v$: $Tv=\lambda v$
• Distinct eigenvalues have independent eigenvectors
• Basis of eigenvectors results in a diagonal matrix for $T$
• Every linear map has an upper triangular matrix representation

## Dot product and length in $\mathbb{R}^2$

$\mathbb{R}^2$: Dot product of $x=(x_1,x_2)$ and $y=(y_1,y_2)$

$\langle x,y \rangle=x_1y_1+x_2y_2\in\mathbb{R}$

• Fix $y$: linear in $x$

• Related to angle between $x$ and $y$

$\mathbb{R}^2$: norm of $x=(x_1,x_2)$

$\lVert x\rVert=\sqrt{\langle x,x\rangle}=\sqrt{x^2_1+x^2_2}\in\mathbb{R}^+$

• Nonlinear, but closely connected to dot product

• Length or magnitude of $x$ or distance from origin

## Dot product and length in $\mathbb{C}^n$

$\mathbb{C}^n$: Dot product of $x=(x_1,\ldots,x_n)$ and $y=(y_1,\ldots,y_n)$

$\langle x,y \rangle=x_1\overline{y_1}+\cdots+x_n\overline{y_n}\in\mathbb{C}$

• Fix $y$: linear in $x$

• Conjugation in the second argument

• Generalize notion of angle to $n$ dimensions

$\mathbb{C}^n$: norm of $x=(x_1,\ldots,x_n)$

$\lVert x\rVert=\sqrt{\langle x,x\rangle}=\sqrt{\lvert x_1\rvert^2+\cdots+\lvert x_n\rvert^2}\in\mathbb{R}^+$

• Nonlinear, but closely connected to dot product

• Conjugation in the second argument is important

• Length/magnitude in $n$ dimensions

## Inner product

$V$: vector space over $F=\mathbb{R}$ or $\mathbb{C}$

Inner product on $V$: function mapping two vectors $u,v\in V$ (in that order) to a scalar $\langle u,v\rangle\in F$ satisfying the following properties.

1. positivity: $\langle v,v\rangle\ge0$

2. definiteness: $\langle v,v\rangle=0$ if and only if $v=0$

3. additivity in first argument: $\langle u_1+u_2,v\rangle=\langle u_1,v\rangle+\langle u_2,v\rangle$

4. homogeneity in first argument: $\langle \lambda u,v\rangle=\lambda\langle u,v\rangle$, $\lambda\in F$

5. conjugate symmetry: $\langle u,v\rangle=\overline{\langle v,u\rangle}$

## Examples: $\mathbb{R}^n$ and $\mathbb{C}^n$

Dot product in $\mathbb{R}^n$ and $\mathbb{C}^n$

Yes

$\langle x,y\rangle=x_1y_1+2x_2y_2$ in $\mathbb{R}^2$

Yes

$\langle x,y\rangle=x_1y_1-3x_1y_2-3x_2y_1+20x_2y_2$ in $\mathbb{R}^2$

Yes

$\langle x,y\rangle=x_1y_1-3x_1y_2-3x_2y_1+9x_2y_2$ in $\mathbb{R}^2$

No

## Examples: Functions form $[0,1]$ to $\mathbb{R}$

$\langle f,g\rangle=\int_{0}^1 f(x)g(x)dx$

Yes

$\langle f,g\rangle=\int_{0}^1 f(x)g(x)e^{-x}dx$

Yes

$\langle f,g\rangle=\int_{0}^1 f(x)g(x-1/2)dx$

No

## Inner product space and properties

$V$: vector space over $F$ is called an inner product space if there is a valid inner product defined on $V$.

Properties

1. Fix $u\in V$. Then, $Tv=\langle v,u\rangle$ defines a linear map.

2. $\langle u,0\rangle=\langle 0,u\rangle=0$

3. $\langle u,v_1+v_2\rangle=\langle u,v_1\rangle+\langle u,v_2\rangle$

4. $\langle u,\lambda v\rangle=\overline{\lambda}\langle u,v\rangle$

## Orthogonality and orthogonal decomposition

$V$: inner product space

$u,v$ are orthogonal if $\langle u,v\rangle=0$

Orthogonal decomposition

For $u,v\in V$ with $v\ne0$, there exists $c$ such that $\langle u-cv,v\rangle=0$.

$c=\langle u,v\rangle/\langle v,v\rangle$

$u = \dfrac{\langle u,v\rangle}{\langle v,v\rangle}v + w$

• $cv$: parallel to $v$

• $w=u-cv$: orthogonal to $v$

## Cauchy-Schwarz inequality

$V$: inner product space and $u,v\in V$. Then $|\langle u,v\rangle|^2\le \langle u,u\rangle\langle v,v\rangle$

Proof

• If $v=0$, done

• If $v\ne0$, orthogonal decomposition

$u = \dfrac{\langle u,v\rangle}{\langle v,v\rangle}v + w$, and $\langle v,w\rangle=0$

\begin{align} \langle u,u\rangle&=\dfrac{|\langle u,v\rangle|^2}{\langle v,v\rangle^2}\langle v,v\rangle+\langle w,w\rangle\\ &\ge \dfrac{|\langle u,v\rangle|^2}{\langle v,v\rangle} \end{align}

## Norm

$V$: vector space over $F=\mathbb{R}$ or $\mathbb{C}$

Norm on $V$: function mapping vector $v\in V$ to a scalar $\lVert v\rVert\in \mathbb{R}$ satisfying the following properties.

1. positivity: $\lVert v\rVert\ge0$

2. definiteness: $\lVert v\rVert=0$ if and only if $v=0$

3. absolute scalability: $\lVert \lambda v\rVert=\lvert \lambda\rvert \lVert v \rVert$

4. triangle inequality: $\lVert u+v \rVert\le \lVert u \rVert + \lVert v \rVert$

## Examples: $\mathbb{R}^n$ and $\mathbb{C}^n$

Euclidean or 2-norm or square norm

$\lVert x\rVert=\sqrt{\lvert x_1\rvert^2+\cdots+\lvert x_n\rvert^2}$

1-norm or Manhattan norm or taxicab norm

$\lVert x\rVert=\lvert x_1\rvert+\cdots+\lvert x_n\rvert$

$\infty$-norm or max norm

$\lVert x\rVert=\max_{k=1,\ldots,n} \lvert x_k \rvert$

$p$-norm, $p\ge 1$

$\lVert x\rVert=\left(\lvert x_1\rvert^p+\cdots+\lvert x_n\rvert^p\right)^{1/p}$

## Norm from inner product

$V$: inner product space. $\lVert v\rVert=\sqrt{\langle v,v\rangle}$ is a norm.

Cauchy-Shwarz: $\lvert\langle u,v\rangle\rvert\le \lVert u\rVert\lVert v\rVert$

Proof for triangle inequality

\begin{align} \lVert u+v\lVert^2 &= \langle u+v,u+v\rangle\\ &= \langle u,u\rangle + \langle u,v\rangle + \langle v,u\rangle + \langle v,v\rangle\\ &= \langle u,u\rangle + 2 \text{Re}\langle u,v\rangle + \langle v,v\rangle\\ &\le \langle u,u\rangle + 2 \lvert\langle u,v\rangle\rvert + \langle v,v\rangle\\ &\le \lVert u\rVert^2+2\lVert u\rVert\lVert v\rVert+\lVert v\rVert^2\\ &=(\lVert u\rVert+\lVert v\rVert)^2 \end{align}

2-norm: from dot product

$p$-norm for $p\ne2$: not from any inner product

## Two geometric results

$V$: inner product space and norm defined from inner product

Pythogorean law: if $u,v\in V$ are orthogonal, $\lVert u+v\rVert^2=\lVert u\rVert^2+\lVert v\rVert^2$

Parallelogram law: if $u,v\in V$, $\lVert u+v\rVert^2+\lVert u-v\rVert^2=2(\lVert u\rVert^2+\lVert v\rVert^2)$

Exercise: A norm is from an inner product if and only if it satisfies the parallelogram law.