Resnet V1 vs Resnet V2: Compare the Structure – Deep Learning Tutorial

By | February 8, 2023

In order to increase the depth of a neural networks, we can use resnet networks. There are two kinds of resnet structure, Resnet V1 and Resnet V2. How about their structure and which one we should use? In this tutorial, we will discuss this topic.

Look at 5 types of resnet block structures below.

Resnet V1 vs Resnet V2 - Compare the Structure - Deep Learning Tutorial

If conv(weight)+BN+ReLU is the basic operation, we can find:

original = resnet v1

full pre-activation = resnet v2

How about their performance?

From paper: Identity Mappings in Deep Residual Networks, we can find:

the performance of resnet v1 and resnet v2

  • full pre-activation (resnet v2) has the best performance.
  • (b) and (c) has the bad performance.


  • the last BN in (b) has change the distribution of \(x_l\)
  • the ReLU in (c) make all value is positive.

We also can find:

In order to use resent correctly, we should:

(1) Do not change the value of \(x_l\) lightl, unless you think it is necessary.

(2) Do not use BN (batch norm) after addition operation.