Transfer Knowledge to classify the CIFAR 10 dataset
Abstract
In this blog I am going to talk about my experience trying to accomplish a task: implement transfer knowledge to classify the CIFAR 10 dataset getting a validation accuracy of 87% or higher. Observing the difference it can make using different models, as well as different optimizers. At the end I decided to go with the Inception V3 model and the Adam optimizer
Introduction
Maybe you have heard of transfer learning or transfer knowledge, put it in simple words is applying what you or someone has learnt about a topic as the base of a new development, where you don’t have as much time as the first learning took or the same amount of resources.
Currently Machine Learning is having a lot of progress, we can find a lot of readings about different models, and there are different open source models you can use to apply transfer learning, use them as the base for new experiments or new developments.
Materials and Methods
To approach this task of using transfer knowledge to classify the CIFAR 10 dataset, there are 5 models used for model vision.
- VGG-16
- VGG-19
- Inception V3
- Xception
- ResNet-50
I was between VGG-16, ResNet-50 and Inception V3.
I chose Inception V for several reasons based on the table above, its size is the smallest, its top-1 accuracy is the highest as it is its top-5 accuracy.
I used colab with a GPU, is easy to use and is free, and it does not take to long to run the model, then is the perfect tool to use in this scenario, also I install Tensorflow version 1.x to work with keras.
I used the Cifar 10 dataset, of course, this dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
I also scale the images using bilinear interpolation because the model is trained on much larger images than 32 by 32.
Freeze all layers in the model and add some dropout layers to it, also used Adam optimizer to fast things up, then I tried with SGD and kept it, it did a better job than Adam.
Results
In the image below you can see how the model start with the Adam optimizer, It starts with 76% of accuracy.
And ends with a 95% of accuracy a very big change.
But with SGD it start with 80% of accuracy
And ends with 99% of accuracy
Is very clear why I stick with SGD I got better results with it also the gap between train and validation is minor, it takes a little longer but it does the work.
With this plots is easier to see the difference.
- Adam
- SGD
Discussion
The more important thing for me was avoiding overfitting while getting good results, that is why it did not matter if the SGD take a little longer than Adam optimizer with the training Data, and of course the preprocess of the data set helped with this problem also, a bad preprocessing can get us to an unwanted result.
