#native_company# #native_desc#
#native_cta#
  1. Introduction to Machine Learning with TensorFlow.js
  2. Neural Networks
  3. Using a TensorFlow model in inference mode

Using a TensorFlow model in inference mode

Let's get physical

Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.

Buy now on Amazon

In the previous lectures we learned how to create and then train a neural network using TensorFlow, in this lecture we will learn how to use that model in inference mode. Inference refers to the process of using a trained model to make a prediction.

Code

If we look in our start.js file we’ll see a function called inferModel like so:

function inferModel(data) {
  console.log({
    data
  });

  // TODO
}

The function needs to be fleshed out, it gets called by our application when we press the green “Check” button in the application. The application first scales your digit to 28 by 28 pixels and then converts it into a 1D array then called the inferModel function and passes in as data a 1D array of the image.

We want to pass this data to our TensorFlow model and extract it’s prediction, that’s the goal of this function.

First convert data into a Tensor like so:

let inputs = tf.tensor4d(data, [1, 28, 28, 1]);
inputs.print();

Important

The dimensionality of the inputs to the model need to match the dimensionality of the training data.

When we trained the model we passed in training data with this shape tf.tensor4d(data, [55000,28,28,1]) since we were passing in a Tensor that contained 55,000 images. Even though we are now passing in just one image we still need to pass in a 4D tensor so the first parameter is 1 lie so ` [1, 28, 28, 1]`.

Next we take this inputs, pump them into the start of our model and capture the outputs it pumps back out like so:

const output = MODEL.predict(inputs);

output is a Tensor and if you remember is not a single number, we are expecting an array of 10 values each which contains a probability of the image being one of those categories. To extract those values we need to call dataSync like so:

const distribution = output.dataSync();
console.log({
    distribution
});

distribution now contains a standard JavaScript array which we can print out, like so:

distribution: Float32Array(10)
0: 0.015741486102342606
1: 0.0006235429318621755
2: 0.8232827186584473
3: 0.13048334419727325
4: 0.0009772877674549818
5: 0.0022565999533981085
6: 0.00227708974853158
7: 0.001965487375855446
8: 0.021101541817188263
9: 0.0012907495256513357

The first thing to notice about the distribution is that all the numbers add up to 1, that’s because it’s a probability distribution, with this type of model no matter how many categories you use the end probability distribution has to and should add up to 1.

The second thing to notice is the the value for the 2nd index is the highest, it’s 0.82. That’s because the image I drew was of a number 2 so the model predicted correctly.

So there is a little semantic interpretation that needs to happen, the model outputs some numbers and you need to convert that to a real world meaning, since we trained the model with the 2nd index being 2 thats the meaning of this set of outputs.

The final few bits we need to do are just to display this information in the user interface, let’s actually get the predicted number from our array of probabiliities like so:

  const prediction = getPrediction(output)
  console.log({
    prediction
  });

getPrediction is a helper function, you can find it in the helper.js file, it just loops through the array and finds the index with the highest value and returns that prediction.

Before we leave this function let’s remember to clean up our Tensors, we need to do this so we don’t have a memory leak:

inputs.dispose();
output.dispose();

Finally this function needs to return an object with both the prediction and distribution, like so:

  return {
    prediction,
    distribution
  };

Summary

Using a model is much easier than training one in the first place, it’s called using a model in inference mode and involves pumping in data the same dimension of the training data and extracting the outputs. There may have to be some manipulation of the outputs to extract human meaning from the numbers but that depends on the nature of your machine learning model.


Advanced JavaScript

This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today.

Level up your JavaScript now!