Implement Ordinary Least Squares Linear Regression with Scikit-Learn for Beginners – Scikit-Learn Tutorial

Ordinary Least Squares is a simple linear model in scikit-learn, in this tutorial, we will write an example to explain how to implement ordinary least squares linear regression for beginners.

Import libraries

import numpy as np
from sklearn.linear_model import LinearRegression

import numpy as np
from sklearn.linear_model import LinearRegression

Prepare data (X, y)

X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
y = np.array([2, 4, 5, 7])
print("X = ")
print(X)
print("y = ")
print(y)

X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
y = np.array([2, 4, 5, 7])
print("X = ")
print(X)
print("y = ")
print(y)

In this example, we use 4 samples, each sample contains 2 features. where X are samples and y are the true value.

Create ordinary least squares to estimeate w and w_o

reg = LinearRegression().fit(X, y)

reg = LinearRegression().fit(X, y)

We use fit() function to calculate the loss function of ordinary least squares and get w andw_o.

Print w and w_o

coef_ = reg.coef_
print(coef_)
intercept_ = reg.intercept_
print(intercept_)

coef_ = reg.coef_
print(coef_)
intercept_ = reg.intercept_
print(intercept_)

w is containd in reg.coef_ and w_o is containd reg.intercept_. In this example, they are:

[1. 2.]
-0.9999999999999982

[1. 2.]
-0.9999999999999982

Which means each predicted y_pre is:

y_pre = 1*x₁ + 2*x₂ + -0.9999999999999982

How about the qualities of w and w_o

We should calculate r2 coefficient to estimate.

r2 = reg.score(X, y)
print(r2)

r2 = reg.score(X, y)
print(r2)

The r2 coefficient is 1.0, which means the qualities of wandw_o are very good, they can fit the true value very well.

How to prodict by X, w and w_o

We can use X, w and w_o to predict a value.

y_predict = reg.predict(np.array([[3, 5]]))
print(y_predict)

y_predict = reg.predict(np.array([[3, 5]]))
print(y_predict)

The predict value is: 12