What is Transfer Learning?

Think of the type of Machine Learning we have been doing so far as teaching a person JavaScript from scratch. Transfer Learning like teaching JavaScript to a Python developer.

It’s the transfer of knowledge from two related domains. As humans, we do this all the time. For example, if you can ride a motorbike, you have at least some of the knowledge needed to learn how to drive a car. If you are very good at playing the piano, you will probably find it easier to learn how to play the guitar than someone who has never played any instrument.

An excellent explanation for why this is so powerful is when we dig into how a CNN works.

With Convolutional Neural Networks (CNN) we create convoluted features that are trained to highlight certain features in images. We can work backward and give a model an image and have it tell us what each of those convoluted features highlights in that image.

If you were to example the internal layers of a CNN that has learned how to classify faces. You might see how each layer has learned how to highlight different parts of the input image; the hidden layers learn to recognize more sophisticated features.

For a thorough reading on the internal workings of a CNN, I recommend reading the paper Visualizing and Understanding Convolutional Networks^[1].

If we wanted to create an app that recognizes specific hand signals, we could start with a labeled data set of pictures of people making hand signals and the associated meaning (label) and then train the model from scratch. What if instead of starting from scratch, we start with a model that already knows how to recognize other things in images and then retrain it to recognize hand signals?

The MobileNet model we used in the first application we built is an ideal model to retrain. It can recognize 1000 things in the world; the hidden layers have learned to highlight things like lines, curves, and other features. It’s only the last layer that pulls this information together and makes a classification into one of the 1000 things.

In transfer learning, we essentially decapitate the source model, which means we strip off the last few layers, and then attach onto it a new fresh model. During training, we may choose to retrain the source model, or more commonly, we leave those weights as is and train the new model we attached to it.

We start with a pre-trained model, like so:

We decapitate it, which means we strip off the last layer and that the outputs from a previous layer as the inputs to a new, untrained model, like so:

Through this approach, we have a few advantages:

In this chapter, we will build a hand-sign recognizer application that uses transfer learning with a base decapitated MobileNet model with a new type of classifier attached to it called a KNN model. More on that in the next lecture.