TensorFlow tf.sqrt() function may occur NaN error. In this tutorial, we will use a simple tip to fix it.
tf.sqrt() function is widely used for computing distances. For example cosine distance and euclidean distance. We also often use these distances as loss funcitons when training our model. However, it may get NaN value, which means these loss functions are also nan.
This image is an example for using euclidean distance to be a part of loss function.
To fix NaN error for computing cosine distance, you can read this tutorial.
TensorFlow Calculate Cosine Distance without NaN Error
How to void NaN in tf.sqrt()
We can limit the value of tf.sqrt(). Here is an example:
tf.sqrt(tf.maximum(d, 1e-9))
Where d is a tensor.
How to fix NaN in euclidean distance？
We can modify euclideanMeanDistance() function.
def euclideanMeanDistance(x, y):
d = tf.reduce_sum(tf.square(x - y), 1)
dx = tf.maximum(d, 1e-9)
dist = tf.sqrt(dx)
return tf.reduce_mean(dist)
Then you will find this function will not report NaN error.