# Understand LSTMP (LSTM with Recurrent Projection Layer): Comparing with LSTM

By | December 14, 2020

LSTMP (LSTM with Recurrent Projection Layer) is an improvement of LSTM with peephole conncections. In this tutorial, we will introduce this model for LSTM Beginners.

## Compare LSTMP and LSTM with with peephole conncections

LSTMP is proposed in paper LONG SHORT-TERM MEMORY BASED RECURRENT NEURAL NETWORK ARCHITECTURES FOR LARGE VOCABULARY SPEECH RECOGNITION.

 LSTM with peephole conncections LSTMP  $$m_t$$ is the output of the current lstm cell. As to equations (1)-(5) and (7)-(11), they are the same.

We usually set $$y_t = m_t$$ in LSTM. However, we add a recurrent projection layer in LSTMP.

This recurrent projection layer is:

$r_t = W_{rm}m_t$

$$r_t$$ is the output of the current lstm cell. We use a weight $$W_{rm}$$ to compress or enlarge $$m_t$$.

For example:

If the shape of $$m_t$$ is 1*200, $$W_{rm}$$ is 200* 100. The shape of $$r_t$$ is 1* 100. However, if the shape of $$W_{rm}$$ is 200* 300. The output of $$r_t$$ will be 1 * 300.