When you are using tensorflow to training your model, you may find an error: the value of loss function is getting bigger and bigger when training. How to solve this problem? In this tutorial, we will discuss two ways to fix this error.
Why the value of loss function is getting bigger and bigger
There are two reasons:
(1) The learning rate is high
(2) The value of hyperparameters are too large
For example:
As to loss function
loss = loss_1
There is only a loss function, you should check your learning rate is high or not. If you have set it to be 0.001, which means the loss function may be wrong, you should modify your model. However, if the original learning rate is 0.005, you can set it to be 0.001
As to loss function
loss = a * loss_1 + b * loss_2
There are two hyperparameters a and b, if the learning rate is low, you should consider the value of a and b is too large or not. Decrease them.