# Understand Orthogonal Regularization in Deep Learning: A Beginner Introduction – Deep Learning Tutorial

By | October 13, 2020

Orthogonal Regularization is a regularization technique which is often used in convolutional neural networks. In this tutorial, we will introduce it for deep learning beginners.

## What is Orthogonal Regularization

There are two types of Orthogonal Regularization, they are:

## L1 Norm Orthogonal Regularization

It is defined as:

## L2 Norm Orthogonal Regularization

where $$I$$ is an identity matrix, $$W$$ should be initialized as an orthogonal matrix.

In tensorflow, in order to create a random orthogonal matrix, you can read:

TensorFlow Create a Random Orthogonal Matrix: A Beginner Guide

If you do not use tensorflow, you can read:

Python Create a Random Orthogonal Matrix: A Beginner Guide

In order to compute matrix L1,L2 norm in tensorflow, you can refer:

TensorFlow Calculate Matrix L1, L2 and L Infinity Norm: A Beginner Guide – TensorFlow Tutorial