How do neural networks learn?

The following two tabs change content below.
Prasad Kharkar is a java enthusiast and always keen to explore and learn java technologies. He is SCJP,OCPWCD, OCEJPAD and aspires to be java architect.

Latest posts by Prasad Kharkar (see all)

We have already learned about neurons and neural networks . Neural networks improve accuracy of predictions, but how do they do it? How do neural networks learn ?

How do Neural Networks Learn?

The Neuron:

The Neuron in Deep Learning

The Neuron in Deep Learning

We have already seen

  • Each neuron can accept inputs and provide output through synapses.
  • Each synapse has some weight
  • Neuron calculates the weighted sum of all inputs and associated weights and then activation function decides its output.

In this process, data is only flowing in forward direction i.e. inputs applied to neuron, neuron performs certain calculation, then activation function triggers the output. However, once output is done, there is no way to improve its accuracy. We can improve it once neural network learns the optimal way for predictions. The question still remains “How do neural networks learn?”.

The Cost Function:

A commonly used cost function is

C = ½(y’-y)2


  • y’ is predicted value for input variables
  • y is actual value from test dataset
  • C is the cost function value.
cost function calculation

cost function calculation

Now, again inputs provided to neuron, it calculates the weighted sum. Activation function is applied to neuron and it predicts output value. This prediction is compared with actual value and cost function is calculated. Still we want to know “How do neural networks learn?”.


Now let us use the cost function we calculated.



  • Cost function is fed to neuron and weights are adjusted.
  • Again, neuron calculates weighted sum with updated weights.
  • Activation function is applied to neuron and it makes another prediction for same data points.
  • Then again, cost function is calculated from new prediction and actual value and it is fed back to neuron.
  • The process goes on until optimum cost function i.e. having value closest to zero is obtained.

A whole neural network can learn just like a neuron learns. It decides optimum cost function based on gradient descent or stochastic gradient descent. However, we will learn about them in different article.



Share Button

1 comment for “How do neural networks learn?

Leave a Reply

Your email address will not be published. Required fields are marked *