Calculating loss

Let's get physical

Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.

Buy now on Amazon

Code
Summary

The first problem we have to deal with in any Neural Network is to figure out how we are going to calculate the loss. The loss needs to be the lowest when our values for a and b are optimal.

Our data is a set of points, of x and y, x is the input and y being the output.

Figure 1. Data set

The best-fit line is made from a series of points with the same values of x as the data points but different values of y, like so:

Figure 2. Data set with best fit line

The x values are the same, but the y values are different. I like to call the y values of our data, actualY, and the y values from our model the predictedY. Our model’s loss is how far our predictedY values are from our actualY values.

Figure 3. Data set with best fit line and error

In this scenario, to calculate the loss, we can use our trusty Mean Square Error again! We take the actualY from the predictedY, square the result, and then get the average.

Code

Open up the loss-regression folder in the demo code folder.

As usual, we have a start.js file and a completed.js file, we will add our code to the start.js file. If you get stuck, check your code against the completed.js file.

If you run the index.html application now, it will bring up a screen with a line across it; clicking on the screen will add a point; however, nothing else will happen. The application isn’t complete.

Figure 4. Loss regression application, can add dots but it doesn’t calculate the best fit line

Note

In the code, we’ve scaled the width and height, so it starts at 0 and finishes at 1. 0 for x means to the far left of the screen, and 1 for x means the far right. Scaling numbers to between 0 and 1 is an excellent habit to get into with Neural Networks; you can always scale them back out to real-world numbers later, but within the confines of a neural network, try to keep the numbers small and contained in the same boundaries as each other.

I’ve hardcoded values of -0.3 for a and 0.5 for c in our line equation. The negative value for a means it slopes down, and the value of 0.5 for c means it crosses the y-axis half-way up the screen.

In the loss-calculation application, we are going to keep the values of a and c fixed, and our job is to figure out the loss of the application given the points we click on the screen.

Let’s take a look at the start.js code first, at the top of the file we have a few variables:

let LOSS = 0; 	(1)

let Xs = []; 	(2)
let Ys = [];

let A = -0.3;	(3)
let C = 0.5;
const calculatePredictedY = x => A * x + C; 	(4)

1	This is a helper variable, anything you store in here will be displayed in the bottom left hand side of the screen.
2	Every-time you click on the screen the mouse x, y points will be appended to these arrays, the Xs in one and the Ys in another.
3	`A` and `C` are the variables that represent our line; if you adjust these, it will change the slope and intercept of the line on the screen.
4	A helper equation, given the values of `A` and `C` if we pass it an `x`, it will calculate `y` for us.

Then we have a train function, like so:

async function train() {
  calculate_loss()
}

Every time you click the mouse on the screen, the application calls the train function; inside that function, we will calculate our loss using TensorFlow.js and store the result in the LOSS variable so that it gets displayed on the screen.

The first thing we need to do is to covert our Xs and Ys data to Tensors, add this to the top of the train function:

const actualXs = tf.tensor(Xs);
const actualYs = tf.tensor(Ys);

Note

We could use tf.tensor1d(Xs, [Xs.length, 1]) to be more explicit, however, to keep things simple, I’ve chosen to leave these out. In your applications, I recommend you keep things explicit.

We will also need to convert the A and C variable to Tensors, like so:

const a = tf.scalar(A);
const c = tf.scalar(C);

We are using tf.scalar here, this is a Tensor for a single value, a single number.

Next up we need to calculate the list of y points for our best fit line, like so:

predictedYs = a.mul(actualXs).add(c);

This applies the equation of a line all the actualXs (y = a*x + c) and stores the results as a 1D Tensor called predictedYs.

Now we have a 1D Tensor of predictedYs and a 1D Tensor of actualYs, we need to calculate the mean squared error between them, like so:

let loss = predictedYs
    .sub(actualYs)
    .square()
    .mean();

Note

We’ve covered mean squared error in-depth in the Tensors lecture.

Finally, we need to extract the actual value from the loss tensor and store in the normal javascript LOSS variable so it will be displayed on the screen, like so:

LOSS = loss.dataSync()[0];

Now when you run the application, you’ll notice that the number on the bottom left updates as you click around the screen. You know it’s working because if you click very close to the line, then the loss will be close to 0. If you click far away from the line, then the loss will be some number >0, give it a try!

The full source for our application looks like so:

// Store the loss in this variable and it will be printed on the screen.
let LOSS = 0;

// Every-time you click on the screen the mouse x,y points will be stored in here.
let Xs = [];
let Ys = [];

// The equation of a line
let A = -0.3;
let C = 0.5;

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  const a = tf.scalar(A);
  const c = tf.scalar(C);

  predictedYs = a.mul(actualXs).add(c);

  let loss = predictedYs
    .sub(actualYs)
    .square()
    .mean();

  LOSS = loss.dataSync()[0];
}

Summary

Before we start to train a neural network, we need to define a loss function, a way of knowing how wrong our neural network is.

We calculate the loss as the mean square error of the actualY values of the points you click on the screen and their predictedY equivalents on the hardcoded best-fit line.

Although we are making our data by clicking on the screen, we would use the same approach when applying linear regression to another data set.

Advanced JavaScript

This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today.

Level up your JavaScript now!

[🌲,🌳,🌴].push(🌲)

If you find my courses useful, please consider planting a tree on my behalf to combat climate change. Just $4.50 will pay for 25 trees to be planted in my name. Plant a tree!