Calculate Singular Value Decomposition (SVD) using Numpy – Numpy Example

Singular value decomposition(SVD) is an important theory in machine learning, it can decompose a matrix to the product of three matrices:

where:

S is singular value of matrix A.

To caculate S of A, here we write an example using numpy.

import numpy as np
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
U,S,VT = np.linalg.svd(A)
print (U)
print (S)
print (VT)

The result is:

[[-0.21483724  0.88723069  0.40824829]
 [-0.52058739  0.24964395 -0.81649658]
 [-0.82633754 -0.38794278  0.40824829]]
[  1.68481034e+01   1.06836951e+00   1.47280825e-16]
[[-0.47967118 -0.57236779 -0.66506441]
 [-0.77669099 -0.07568647  0.62531805]
 [ 0.40824829 -0.81649658  0.40824829]]

Notice:

1.Singular Value S is ordered from big to small.

2.VT is transposed, you can caculate A as:

Here is an example:

>>> import numpy as np

>>> A = np.mat([[1,2,3,4],[5,6,7,8],[2,3,4,5]])
>>> u, s, v = np.linalg.svd(A)

>>> print(M.shape)
(3, 4)
>>> print(u.shape)
(3, 3)
>>> print(s.shape)
(3,)   
>>> print(v.shape)
(4, 4)

>>> print(u.dot(np.column_stack((diag(s), np.zeros(3))).dot(v)))
[[ 1.  2.  3.  4.]
 [ 5.  6.  7.  8.]
 [ 2.  3.  4.  5.]]