Can Apply a Dropout Layer to Softmax Layer in Neural Networks – Deep Learning Tutorial

By | October 28, 2019

Softmax layer is often used as a output layer of a neural networks model. A standard structure of neural networks model with softmax as output is:

But to void overfitting in neural networks, can we apply a dropout layer to softmax layer?

The answer is not if softmax is the output layer.

Look at image below:

If you apply a dropout to softmax layer, you may get only two output not five. As to loss function, less output will minimum the loss function of model.

In order to use dropout layer to prevent overfitting in neural networks, you should use it in front of ouput layer.

Leave a Reply

Your email address will not be published. Required fields are marked *