Understand Singular Value Decomposition (SVD): A Beginner Guide – Deep Learning Tutorial

Singular Value Decomposition (SVD) expresses any matrix can be decomposed into the product of three matrices. In this tutorial, we will discuss some basic features of svd for deep learning beginners.

As a matrx A(m × n), it can be decomposed into the product of three matrices.

where:

UU^T = I, VV^T = I

I is unit matrix.

U^T is transpose matrix of U, V^T is transpose matrix of V.

When we are computing svd value of matrix A, we often sort the value of Σ from big to small, which means:

σ1>= σ2 >= σ3 >= σ4 …….., σi is called singular value of matrix A.

Notice: when matrix A is m × m. A can be decomposed into:

where QQ^T = I

Here are some features of SVD:

if A = UΣV^T

then:

A^T = VΣU^T

A^TA = VΣ²V^T

To compute U, Σ, V, you can use tensorflow or numpy, here are tutorials.

Compute Singular Value Decomposition (SVD) with TensorFlow – TensorFlow Example

Calculate Singular Value Decomposition (SVD) using Numpy – Numpy Example