Understand Bias in Neural Network: Why Using Bias in Neural Network

By | July 6, 2020

In neural network, we should use a bias, for example:

y = w•x + b

where b is the bias.

Why we need use a bias in neural network? In this tutorial, we will answer this question.

Look at image below:

bias in neural network

There are two kinds of points in 2 dimension space. In order to separate them, we can use a linear function.

y = w•x + b

Bias b must be need, otherwise, we will not seperate these two kinds of points correctly.

However, if we set b = 0, which mean we do not use bias, what problem will occur?

It means y = w•x

In order to sperate these two kinds of point correctly, we should move all points.

bias in neural network - move points

Which means

y’ = y – b

However, it is very hard or impossible to move sample data to a specific point to make bias = 0.  Especially there are some noise data in your training data. So in order to increase the accuracy of neural network, we should use a bias.

Leave a Reply