TensorFlow LSTM Implements L2 Regularization: A Practice Guide – TensorFlow Tutorial

By | November 16, 2019

LSTM neural network is widely used in deep learning, tensorflow also provides some lstm classes. However, these classes look like some black boxes for beginners. How to regularize them? In this tutorial, we will discuss how to add l2 regularization for lstm network.

lstm l2 regularization

To regularize a lstm network, we have two methods:

1. Add a dropout for lstm

2.Implement l2 regularization for lstm

In this post, we will talk about how to add l2 regularizationo for lstm.

List all trainable weights in lstm

We only need to know all trainable weights in lstm, then we can apply l2 regularization on these weights. To list these

weights, we can refer to this tutorial.

Get LSTM Cell Weights and Regularize LSTM in TensorFlow

Implement l2 regularization on trainable weights in lstm

After getting trainable weights in lstm, we can use tf.nn.l2_loss() function to implement l2 regularizaition. Meanwhile, if you use a lstm in a complex neural network, to regularize it, you can refer to this tutorial.

Multi-layer Neural Network Implements L2 Regularization in TensorFlow