Math and science::Algebra

Singular value decomposition

Singular value decomposition (SVD)

Let A be an $m \times n$ matrix. SVD decomposes $A$ as:

A = U Σ V^{T}

where $U$ and $V$ are orthogonal matrices and $Σ$ is a diagonal matrix with non-negative real numbers on the diagonal. The diagonal entries of $Σ$ are called the singular values of $A$ .

Assuming this decomposition of $A$ is possible, how can we find $U$ , $Σ$ , and $V$ ?

What is a geometric interpretation of the SVD?

Finding $U$ , $Σ$ , and $V$

Let $A$ be an $m \times n$ matrix. Then:

A^{T} A = (V Σ^{T} U^{T}) (U Σ V^{T}) = V Σ^{2} V^{T}

So, $V$ is the othogonal eigenvector matrix that diagonalizes the symmetric matrix $A^{T} A$ .

Similarly:

A A^{T} = (U Σ V^{T}) (V Σ^{T} U^{T}) = U Σ^{2} U^{T}

So, $U$ is the orthogonal eigenvector matrix that diagonalizes the symmetric matrix $A A^{T}$ .

In both cases, we can read off $Σ$ by taking the square root of the diagonal entries of the diagonalized $A^{T} A$ or $A A^{T}$ .

Singular value decomposition

Singular value decomposition (SVD)

Finding U, Σ, and V

Geometric interpretation

Finding $U$ , $Σ$ , and $V$