\( \newcommand{\matr}[1] {\mathbf{#1}} \newcommand{\vertbar} {\rule[-1ex]{0.5pt}{2.5ex}} \newcommand{\horzbar} {\rule[.5ex]{2.5ex}{0.5pt}} \newcommand{\E} {\mathrm{E}} \)
deepdream of a sidewalk
Show Question
\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \)
Math and science::Algebra

Singular value decomposition

Singular value decomposition (SVD)

Let A be an \( m \times n \) matrix. SVD decomposes \( A \) as:

\[ A = U \Sigma V^T \]

where \( U \) and \( V \) are orthogonal matrices and \( \Sigma \) is a diagonal matrix with non-negative real numbers on the diagonal. The diagonal entries of \( \Sigma \) are called the singular values of \( A \).

Assuming this decomposition of \( A \) is possible, how can we find \( U \), \( \Sigma \), and \( V \)?

What is a geometric interpretation of the SVD?


Finding \( U \), \( \Sigma \), and \( V \)

Let \( A \) be an \( m \times n \) matrix. Then:

\[ A^T A = (V \Sigma^T U^T) (U \Sigma V^T) = V \Sigma^2 V^T \]

So, \( V \) is the othogonal eigenvector matrix that diagonalizes the symmetric matrix \( A^T A \).

Similarly:

\[ A A^T = (U \Sigma V^T) (V \Sigma^T U^T) = U \Sigma^2 U^T \]

So, \( U \) is the orthogonal eigenvector matrix that diagonalizes the symmetric matrix \( A A^T \).

In both cases, we can read off \( \Sigma \) by taking the square root of the diagonal entries of the diagonalized \( A^T A \) or \( A A^T \).

Geometric interpretation