LSTM neural network is widely used in deep learning, tensorflow also provides some lstm classes. However, these classes look like some black boxes for beginners. How to regularize them? In this tutorial, we will discuss how to add l2 regularization for lstm network.
To regularize a lstm network, we have two methods:
1. Add a dropout for lstm
2.Implement l2 regularization for lstm
In this post, we will talk about how to add l2 regularizationo for lstm.
List all trainable weights in lstm
We only need to know all trainable weights in lstm, then we can apply l2 regularization on these weights. To list these
weights, we can refer to this tutorial.
Get LSTM Cell Weights and Regularize LSTM in TensorFlow
Implement l2 regularization on trainable weights in lstm
After getting trainable weights in lstm, we can use tf.nn.l2_loss() function to implement l2 regularizaition. Meanwhile, if you use a lstm in a complex neural network, to regularize it, you can refer to this tutorial.
Multi-layer Neural Network Implements L2 Regularization in TensorFlow