#native_company# #native_desc#
#native_cta#
  1. Introduction to Machine Learning with TensorFlow.js
  2. Regression
  3. Linear Regression Application

Linear Regression Application

Let's get physical

Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.

Buy now on Amazon

In this lecture, we will build an application that does Linear regression using TensorFlow.js in the browser.

So far we’ve covered an introduction to regression, an explanation of linear regression and we learned how to calculate loss in a linear regression app. Now we will build the rest of the application, which learns the best-fit-line for our data.

Code

Open up the linear-regression folder in the demo code folder.

As usual, we have a start.js file and a completed.js file, we will add our code to the start.js file. If you get stuck, check your code against the completed.js file.

Note

=== There is also a ui.js file, this is the user interface code, which will do the work of drawing lines and text on the screen. We won’t be touching the ui.js file or explaining its contents. We are using a library called p5.js[1] to create our user interface. ===

Settings and Variables

Let’s first discuss some of the javascript variables at the top of the start.js file:

// Variables and constants
let LOSS = 0;             (1)
let CURRENT_EPOCH = 0;    (2)
const MAX_EPOCHS = 300;   (3)

// This will store mouse x,y points that have been scaled from 0->1
let Xs = [];              (4)
let Ys = [];              (4)

// The equation of a line
let A = Math.random();    (5)
let C = Math.random();    (5)
1 The current loss of the model, whatever we store in here will be displayed in the bottom left hand corner of the screen.
2 This current epoch, how many times we have run the training on the model. It is displayed in the bottom right-hand corner of the screen.
3 The max iterations we will try to train the model for each new data point added.
4 The variables that store the X and Y points.
5 The variables that represent a line, we initialize them with random numbers.

Training and Loss

Next, let’s look at the train function, this gets called every time a mouse is clicked on the screen, and a new data point is added to our collection.

This contains the code from the Calculating Loss lecture, that’s a good starting point, but we will need to update this to work in our current application.

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  const a = tf.scalar(A);
  const c = tf.scalar(C);

  predictedYs = a.mul(actualXs).add(c);

  let loss = predictedYs
    .sub(actualYs)
    .square()
    .mean();

  LOSS = loss.dataSync()[0];
}

First, let’s extract the a and c variables and move them to the top of the file. They only need to be created once, but inside the train function, they are created every time the button is clicked, so remove them from the train function.

It should now look like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  predictedYs = a.mul(actualXs).add(c);

  let loss = predictedYs
    .sub(actualYs)
    .square()
    .mean();

  LOSS = loss.dataSync()[0];
}

Creating an optimizer

We also need to create an optimizer so let’s create that as well. At the top of the file under the other variables add this code:

const a = tf.variable(tf.scalar(A));
const c = tf.variable(tf.scalar(C));

const learningRate = 0.5;
const optimizer = tf.train.sgd(learningRate);

Note

We are being a lot more explicit now, instead of using tf.variable(A) we are using tf.variable(tf.scalar(A)). It didn’t matter before because we were not using the values of a and c outside, so we didn’t care how TensorFlow treated it, but this time we do care, we want TensorFlow to treat it as a single value, so we tell it it’s a single scalar value.

Tip

We’ve covered the concept of learning rates and optimizers before; for a refresher, please read the lecture on Optimization. In our app, we are using the TensorFlow.js stochastic gradient descent[2] optimizer.

Now we have created the optimizer let’s use it to optimise the a and c variables. The optimizer has a minimize function that expects to input a function that returns the current loss of the model. We’ve got the loss calculation already in our train function so let’s just wrap it with optimizer.minimize(() ⇒ { …​ }); like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  optimizer.minimize(() => {
    const predictedYs = a.mul(actualXs).add(c);
    let loss = predictedYs
      .sub(actualYs)
      .square()
      .mean();

    LOSS = loss.dataSync()[0];
    return loss;
  });
}

This will run one training run across all the data points and calculate new, more optimal values for a and c. If you run this you wouldn’t see the line change on the screen, to change the line drawn on the screen we need to update the A and C variables, we can use dataSync() function, like so:

A = a.dataSync()[0];
C = c.dataSync()[0];

The final state of our train function looks like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  optimizer.minimize(() => {
    const predictedYs = a.mul(actualXs).add(c);
    let loss = predictedYs
      .sub(actualYs)
      .square()
      .mean();

    LOSS = loss.dataSync()[0];
    return loss;
  });
  A = a.dataSync()[0];
  C = c.dataSync()[0];
}

Training for multiple iterations

If we run our application, as you click on the screen and add data points, you will see the line change. Every time you click, TensorFlow.js is doing one single training iteration and adjusting the best fit line just from that.

What if we run multiple training runs with exactly the same data, can we get better results? Let’s wrap the optimizer code with a loop to see, like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  for (CURRENT_EPOCH = 0; CURRENT_EPOCH < MAX_EPOCHS; CURRENT_EPOCH++) {
    optimizer.minimize(() => {
      const predictedYs = a.mul(actualXs).add(c);
      let loss = predictedYs
        .sub(actualYs)
        .square()
        .mean();

      LOSS = loss.dataSync()[0];
      return loss;
    });
    A = a.dataSync()[0];
    C = c.dataSync()[0];
  }
}

await tf.nextFrame

Every time we click, there is a short pause, and things seem to lock up, after about 10 seconds on my computer, I finally see the dot and the best-fit-line appear. The reason for this is that the computer is busy doing all the machine learning training, it’s taken complete control, and the browser doesn’t have a chance to update the screen until the training has finished.

To give the browser a chance to update the screen, we can use the tf.nextFrame function. This gives a break to the training and lets the browser update before continuing the training again. It returns a Promise that resolves when it’s time to continue painting, so we use it together with the JavaScript await function like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  for (CURRENT_EPOCH = 0; CURRENT_EPOCH < MAX_EPOCHS; CURRENT_EPOCH++) {
    optimizer.minimize(() => {
      const predictedYs = a.mul(actualXs).add(c);
      let loss = predictedYs
        .sub(actualYs)
        .square()
        .mean();

      LOSS = loss.dataSync()[0];
      return loss;
    });
    A = a.dataSync()[0];
    C = c.dataSync()[0];
    await tf.nextFrame() (1)
  }
}
1 await tf.nextFrame gives the browser a chance to paint the screen.

tf.tidy

We are creating many Tensors in our application. As discussed in the Optimization lecture Tensors are not automatically deleted in JavaScript unlike standard JavaScript variables, we need to delete them ourselves, or we get a memory leak.

We could call dispose() on all the Tensors at the end of the loop, or we could call tf.tidy like so:

async function train() {
  const actualXs = tf.tensor(Xs);
  const actualYs = tf.tensor(Ys);

  for (CURRENT_EPOCH = 0; CURRENT_EPOCH < MAX_EPOCHS; CURRENT_EPOCH++) {
    tf.tidy(() => { (1)
      optimizer.minimize(() => {
        const predictedYs = a.mul(actualXs).add(c);
        let loss = predictedYs
          .sub(actualYs)
          .square()
          .mean();

        LOSS = loss.dataSync()[0];
        return loss;
      });
      A = a.dataSync()[0];
      C = c.dataSync()[0];
    })
    await tf.nextFrame();
  }

  actualXs.dispose(); (2)
  actualYs.dispose();
}
1 tf.tidy is useful when dealing with lots of tensors that are automatically created due to operations, but can have issues when dealing with async/await and promises so best used wrapped around small sections of code.
2 We can also use the dispose function, it has the added advantage of making the code very clear and easy to understand.

Summary

We put lots of learnings together in this lecture to build our application. We used the loss calculation function we figured out in the Calculating loss lecture, we combined it with the learnings from the Optimization lecture and built our first machine learning model that does training.

Note

Please take a moment to breathe and congratulate yourself; it took real courage and dedication to get to this point!

You learned what regression is and how to build and train a linear regression model using TensorFlow.js. This demo app uses data you generated from mouse clicks, but you could just as easily use the same model to calculate the best fit line for other fixed data sets.


Advanced JavaScript

This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today.

Level up your JavaScript now!