#native_company# #native_desc#
#native_cta#
  1. Introduction to Machine Learning with TensorFlow.js
  2. Neural Networks
  3. Creating a densely connected Neural Network

Creating a densely connected Neural Network

Let's get physical

Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.

Buy now on Amazon

We’re ready to build our Neural Network, which will learn to recognize hand-drawn digits. We are going to start with a simple model, that also happens to work surprisingly well, called a densely connected neural network.

To explain this, let’s work with a simplified example, like so:

5.mnist.003
Figure 1. An example of a three-layer densely connected neural network

We have three layers, one input layer of 4 nodes, one output layer of 2 nodes, and one hidden layer of 3 nodes. Each of the black lines are weights, which we train and use to transform the inputs to the outputs.

The reason it’s densely connected is that a weight connects each node to each node in the next layer, so each node has an input from every single node in the previous layer.

We could write this using the TensorFlow Core API the same way we used in the previous demo application, it’s possible but would require much boilerplate code.

Instead we are going to start using the TensorFlow Layers API, this abstracts us from the lower level implementation details and allows us to more easily build neural networks which are composed of layers of neurons. In fact to represent the simple neural network above only requires 4 lines of code, like so:

const model = tf.sequential(); (1)
model.add(tf.layers.flatten({ inputShape: [4, 1] })); (2)
model.add(tf.layers.dense({ units: 3,activation: "relu" })); (3)
model.add(tf.layers.dense({ units: 2,activation: "softmax" })); (4)
1 We use the TensorFlow helper function to create a sequential model.
2 We add the first layer, which takes as input four values (4 rows of one value each), flatten forces the inputs into a 1D array.
3 We add our hidden layer of 3 nodes. We also define an activation function of relu. This is just a scaling function that determines what number the node emits as an output. We discussed these in the starting What is a Neural Network? lecture. dense means we also create weights between each node in the previous layer and node in this layer.
4 We add our output layer of 3 nodes, this time we use another activation function called softmax, we will discuss this in more detail later.

As you can see, creating Neural Networks with the TensorFlow Layers API is much easier than with the Core API, however it’s essential to understand that what’s going on underneath is the same as with the Core API. The framework is still creating nodes with weights represented as Tensors and Tensor Variables, and later on, we will still be using a loss function with an optimizer. The key point here is that a lot is now abstracted away for you into a line of code.

Completing our code

The example above was for a simple four input and two output Neural Network. For our MNIST example application, we need to have 784 input nodes, one for each pixel value of a 28 x 28 x 1 image of a digit and we need a 10 node output layer, one for each potential value the image could be out of the set “0,1,2,3,4,5,6,7,8,9”.

The code changes are minimal however, inside the createDenseModel() function in the start.js file just write this code:

  const model = tf.sequential();
  model.add(tf.layers.flatten({
    inputShape: [IMAGE_H, IMAGE_W, 1] (1)
  }));
  model.add(tf.layers.dense({ (2)
    units: 42,
    activation: "relu"
  }));
  model.add(tf.layers.dense({ (3)
    units: 10,
    activation: "softmax"
  }));
  return model; (4)
1 IMAGE_H and IMAGE_W are both 28, from our previous lecture you’ll recognize this as the shape of our input data, we need to take this and flatten it to a 1D input layer of 784 nodes.
2 We create a 42 node dense hidden layer, why 42? Why not! Pick another see if it works better :)
3 The final output layer needs to be 10 nodes. softmax is an activation function which turns a set of numbers into a probability distribution, since we are building a classifier and using the categoricalCrossentropy loss function we need the output to be a probability distribution. Details like this are in the TensorFlow API docs[1] for each loss function.
4 Remember to return the model you’ve created.

Summary

You’ve now created your first Neural Network. It doesn’t work yet; we still have some more code to write but well done for getting here!

We learned what a densely connected Neural Network is, and we created one using the TensorFlow Layers API. In the next lecture, we will cover how to train this model using the training data we’ve already prepared.


Advanced JavaScript

This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today.

Level up your JavaScript now!