Orthogonal Regularization is a regularization technique which is often used in convolutional neural networks. In this tutorial, we will introduce it for deep learning beginners.

**What is Orthogonal Regularization**

There are two types of Orthogonal Regularization, they are:

**L1 Norm Orthogonal Regularization**

It is defined as:

**L2 Norm Orthogonal Regularization**

where \(I\) is an identity matrix, \(W\) should be initialized as an orthogonal matrix.

In tensorflow, in order to create a random orthogonal matrix, you can read:

TensorFlow Create a Random Orthogonal Matrix: A Beginner Guide

If you do not use tensorflow, you can read:

Python Create a Random Orthogonal Matrix: A Beginner Guide

In order to compute matrix L1,L2 norm in tensorflow, you can refer:

TensorFlow Calculate Matrix L1, L2 and L Infinity Norm: A Beginner Guide – TensorFlow Tutorial