TensorFlow tf.norm() function is often used to calculate the norm of vector and matrix. However, this function is not a good way to compute the norm of matrix. In this tutorial, we will use some examples to show you this truth.
Syntax
tf.norm() is defined as:
tf.norm(
tensor,
ord='euclidean',
axis=None,
keepdims=None,
name=None,
keep_dims=None
)We can use this function to compute l0, l1, l2 and l infinity norm of a vector and matrix by ord and axis.
Parameters
ord: Order of the norm. It can be 1, 2, np.inf
axis: axis can be None, integer. It determines how to compute vector norm on which axis.
You should notice: if axis = None, tf.norm() will flatten tensor then comput norm.
We will use some examples to introduce how to use tf.norm().
Preliminary
In order to compute the norm of vecters, you should know what is vector norm and how to compute. Here is a tutorial.
Understand Vector Norm: A Beginner Introduction
Compute l1 norm when axis = None
Look at example code below.
import tensorflow as tf import numpy as np xs = tf.Variable(np.array([[1, 2],[3, 4]]), dtype = tf.float32) l1 = tf.norm(xs, axis = None)
xs is a 2*2 tensor, it contains two vectors. We will compute its l1 norm.
init = tf.global_variables_initializer()
init_local = tf.local_variables_initializer()
with tf.Session() as sess:
sess.run([init, init_local])
print(sess.run(l1))
Run this code, you will get the l1 norm of xs is: 10.0
Why the value of l1 norm is 10.0?
Becuase axis = None, tf.norm() will convert 2*2 xs to:[1, 2, 3, 4], then compute the l1 norm of it. The value of l1 norm is 10.0
However, tf.norm() is a good choice to compute the norm of vecters, it is not a good way to compute the norm of matrix. We will use an example to illustrate it.
Compute the l2 norm of vecter xs
We will compute the l2 norm of vecter xs on axis = None, axis = 0 and axis = 1. Here is an example:
l2_none = tf.norm(xs, ord = 2, axis = None)
l2_0 = tf.norm(xs, ord = 2, axis = 0)
l2_1 = tf.norm(xs, ord = 2, axis = 1)
init = tf.global_variables_initializer()
init_local = tf.local_variables_initializer()
with tf.Session() as sess:
sess.run([init, init_local])
print(sess.run(l2_none))
print(sess.run(l2_0))
print(sess.run(l2_1))
Run this code, we will get the result.
| axis | Value |
| None | 5.477226 |
| 0 | [3.1622777 4.472136 ] |
| 1 | [2.236068 5. ] |
If tensor xs is a matrix, the value of its l2 norm is:5.4649854. All value above is not 5.4649854. It means tf.norm() can not calculate the l2 norm of matrix correctly.
In order to know how to compute matrix norm in tensorflow, you can read:
TensorFlow Calculate Matrix L1, L2 and L Infinity Norm: A Beginner Guide – TensorFlow Tutorial
As to tf.norm(), it will compute the l1, l2, l infinity norm based on vector norm algorithm, not matrix norm algorithm. If you plan to use it to compute the matrix norm, you must notice this point.