Training a neural net

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at:, or follow me on Twitter.

A great intro:


Training is how we teach a neural network what we want it to learn. It follows a simple five step process:

1.) Create a training data set, which we will call x and load its labels as targets y

2.) Feed the x data forward through the network with the result being predictions y’

3.) Figure out the “loss” of the network, which is the difference between the predictions y’ and the correct targets y

4.) Compute the “gradient” of the loss (l) and which tells us how fast we’re moving towards or away from the correct targets

5.) Adjust the weights of the network in the opposite direction of the gradient and go back to step two to try again

Now we’ll add some layers.
Most neural networks use fully connected layers. That means they connect every neuron to every other neuron.

Fully connected layers are fantastic for solving all kinds of problems. Unfortunately they don’t scale very well for image recognition.

So we’ll build our system using convolutional layers, which are unique because they don’t connect all the neurons together.

Post external references

  1. 1