An Introduction to Channel-wise Squeeze-Excitation Module (SE Module) – Machine Learning Tutorial

By | June 23, 2022

The Channel-wise squeeze-excitation module (SE module) has achieved a great success in both computer vision and speech processing fields. In this tutorial, we will introduce it for beginners.

SE Module

SE module is proposed in paper: Squeeze-and-Excitation Networks. The structure of it looks like:

A Squeeze-and-Excitation block

The standard SE module uses two fully connected layers to learn the importance of different channels by first compressing and then expanding the full average channel vector to obtain channel level weights.

Here is the detail:

The definition of a Squeeze-and-Excitation block

Effect of SE module

The SE module can use a sigmoid function to compute the attention score for different channels. We also can use it on time sequence or frequency to get which time or frequency is important.