Ridge Regression – A Simple Tutorial for Beginners

By | May 15, 2019

What is ridge regression?

Ridge Regression is a kind of method of linear regression.

What is formula?

Given a input x and output y, you should estimate parameters w and b to calculate ypred to fit y.

It means:

ypred is estimated values by calculating by parameters w and b with input x.

To fit the y by ypred, we should reduce the error between them

Define:

When λ=0, ridge regression will degrade to linear regression function.

Notice: λ ∈[0,∞)

Why should use Ridge Regression?

Ridge Regression can avoid over-fitting problem when training model.

How to set λ value?

We can set it to 0.001 as common, it will be different in different models, it should be set by experiments.

How to estiment w and b?

They can be estimated by SGD algorithm.

Leave a Reply

Your email address will not be published. Required fields are marked *