#native_company# #native_desc#
#native_cta#
  1. Introduction to Machine Learning with TensorFlow.js
  2. Tensors
  3. Tensor Operations

Tensor Operations

Let's get physical

Sometimes, nothing beats holding a copy of a book in your hands. Writing in the margins, highlighting sentences, folding corners. So this book is also available from Amazon as a paperback.

Buy now on Amazon

You can perform operations on Tensors, such as addition, which return new Tensors. TensorFlow.js has an extensive library[1] of available operations, which you are encouraged to review. In this course, we will only be covering the essential operations required to understand the example applications we are building.

Addition

We perform element-wise addition via the add function; this sums two Tensors together using pair wise addition, like so:

3.tensors.001
Figure 1. Image of pairwise addition
var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([4, 0, 1, -9], [2, 2]);
a.add(b).print();

The shape of both the Tensors should be the same to perform element wise addition otherwise it will print an error to the console, like so:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([4, 0, 1], [2, 1]);
a.add(b).print(); // This will print an error

Broadcasting

When the shapes allow, TensorFlow.js will perform an action called broadcasting instead of reporting an error, like so:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([4, 0], [2]);
a.add(b).print();

Even though the shape of a and b are different, the add operation completed successfully and resulted in a Tensor like so:

Tensor
    [[7, 8],
     [8, 6]]

Broadcasting is the process of automatically adjusting matrices to have compatible shapes for arithmetic operations.

Broadcasting means that an operation like so:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([4, 0], [2]);
a.add(b).print();

Was transformed internally into:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([4, 0, 4, 0], [2, 2]);
a.add(b).print();

The data was copied in b, so the shape of b ended up matching the shape of a.

There are specific rules that broadcasting follows, which are documented[2] on the TensorFlow website.

The most common use case, and the use case used in all the examples in this course, is broadcasting with scalars. Doing an addition with any scalar value will result in an element wise addition like so:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.scalar(2);
a.add(b).print();

This results in:

Tensor
    [[5, 10],
     [6, 8 ]]

And is equivalent to:

var a = tf.tensor([3, 8, 4, 6], [2, 2]);
var b = tf.tensor([2, 2, 2, 2], [2, 2]);
a.add(b).print();

Subtraction, Division, and Multiplication

Similar to addition, the sub function performs element wise subtraction between two Tensors. The div function performs element wise division between two Tensors, and the mul function performs element wise multiplication.

Note

The same shape constraints and broadcasting rules apply to subtraction.

Mean Squared Error

Let’s follow on from our Tensor-as-an-Image concept and try to figure out what algorithm we would use to detect if two images are similar to each other?

Let’s imagine two Tensors represent black and white images, like so:

var a = tf.tensor([200, 176, 3, 34], [2,2]);
var b = tf.tensor([213, 132, 0, 98], [2,2]);

Note

These would be some very small images, but the algorithm will work for any size image.

First, we might try subtracting one Tensor from the other. The closer to 0 the resulting Tensor is the closer the images are, like so:

a.sub(b).print()

This results in:

Tensor
    [[-13, 44 ],
     [3  , -64]]

The mean of all the values would give us a single number that defines how close the images are, like so:

a.sub(b).mean().print()

This results in:

Tensor
    -7.5

Comparing the same image with itself returns 0, like so:

a.sub(a).mean().print()

This results in:

Tensor
    0

So it might seem at first glance that this algorithm does a good job, but what if we compared b with a instead of a with b, like so:

b.sub(a).mean().print()

This results in:

Tensor
    7.5

The result here is positive, whereas the result before was negative. That’s not ideal; the order of comparison shouldn’t matter. The main issue here is that the Tensor we are averaging has a mixture of positive and negative numbers like so:

Tensor
    [[-13, 44 ],
     [3  , -64]]

We want all the numbers to be positive, an operation that can help us here is square, if we ran:

a.sub(b).square().print()

This results in:

Tensor
    [[169, 1936],
     [9  , 4096]]

And since all the numbers are now positive then the mean won’t change regardless of the order of comparison:

a.sub(b).square().mean().print()

and

b.sub(a).square().mean().print()

both return:

Tensor
    1552.5

and comparing a with itself returns 0 like so:

a.sub(a).square().mean().print()

returns:

Tensor
    0

In machine learning, you will often have to decide on an algorithm to determine if two things are close to each other. For various reasons, Mean Squared Error is a good algorithm to use, so it’s used frequently throughout this course and in real-life.

Summary

Operations are functions that run on Tensors and return other Tensors. There are many operations.[1] that you can find on the TensorFlow.js website.

In this course, To complete all the example projects, you will only need to know, add, sub, div, mul, mean and square.


Advanced JavaScript

This unique course teaches you advanced JavaScript knowledge through a series of interview questions. Bring your JavaScript to the 2021's today.

Level up your JavaScript now!