Implement Ordinary Least Squares Linear Regression with Scikit-Learn for Beginners – Scikit-Learn Tutorial

By | September 26, 2019

Ordinary Least Squares is a simple linear model in scikit-learn, in this tutorial, we will write an example to explain how to implement ordinary least squares linear regression for beginners.

Ordinary Least Squares example

Import libraries

import numpy as np
from sklearn.linear_model import LinearRegression

Prepare data (X, y)

X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
y = np.array([2, 4, 5, 7])
print("X = ")
print("y = ")

In this example, we use 4 samples, each sample contains 2 features. where X are samples and y are the true value.

Create ordinary least squares to estimeate w and wo

reg = LinearRegression().fit(X, y)

We use fit() function to calculate the loss function of ordinary least squares and get w andwo.

Print w and wo

coef_ = reg.coef_
intercept_ = reg.intercept_

w is containd in reg.coef_ and wo is containd reg.intercept_. In this example, they are:

[1. 2.]

Which means each predicted ypre is:

ypre = 1*x1 + 2*x2 + -0.9999999999999982

How about the qualities of w and wo

We should calculate r2 coefficient to estimate.

r2 = reg.score(X, y)

The r2 coefficient is 1.0, which means the qualities of wandwo are very good, they can fit the true value very well.

How to prodict by X, w and wo

We can use X, w and wo to predict a value.

y_predict = reg.predict(np.array([[3, 5]]))

The predict value is: 12