Mask and Distancing Detector using transfer learning: Part 2

In the last article we completed our face and distance detector, if you haven’t read the article, you can from here. I will write the complete code at the end of this article

Before completing our mask detector we will need to learn about some of the vital topics and which we will be using for our mask detector.

Some of the problem we face while image detection are

1. Lack of data available for model training
2. To train the model from scratch or not for our classification with labeled data.

For problem 1 we have Data augmentaion and for second problem we use Transfer learning.

Data Augmentation

We are cognizant of the fact that image classification requires tons of data, getting the pictures of masked men is a tedious task, so first up we will have to regenerate a lot of data. Before implementing it on Mask data let me try it on some random photo and try to regenerate the photo in large numbers.

from keras.layers import Flatten, Dense
from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array

We will be using Keras API for data recreation or augmentation and use a simple picture of mine. ImageDataGenerator function is used to generate multiple photos from a single photo.

datagen=ImageDataGenerator(rotation_range=40, 
                           width_shift_range=0.2,
                           height_shift_range=0.2,
                           zca_whitening=False, zca_epsilon=1e-06,
                           shear_range=0.4,
                           zoom_range=0.6,
                           horizontal_flip=True,
                           fill_mode="nearest")

Now we would need to specify the location of our photo that we need to regenerate. We will run a for loop that will break after the generation of 20 photos.

img=load_img('image/IMG_20190710_085629.jpg')
x=img_to_array(img)
x=x.reshape((1,)+ x.shape)
i=0
for batch in datagen.flow(x, batch_size=1, save_to_dir='vac_hes/Gaza/generate/augment', save_prefix='anmol',save_format="jpg"):
    i+=1
    if i>20:
        break

The photo that I want to regenerate is my personal photo. I am using this for the illustration.

Here are the regenerated photos in the augment directory. You can generate any number of photos. I have generated 20 photos from this 1 photo.

Now that the data problem is solved, we can move ahead. The next problem however is to start from scratch and train our model or pick up an already trained model on similar data, add our classification model on top of that and use that for our classification. For image classification, we already have labeled data of over a million images called ImageNet and the model is VGG19.

Transfer learning

This process of using an already trained model to solve another similar problem is called transfer learning. For example, a model which is trained for classifying images of different objects like dogs, cats, horses, shoes, etc can be used to detect face mask.

The process of transfer learning can be compared to human brains. A person who already knows how to ride a bicycle and a bike will find it easier to drive a car than a person who has never driven an automobile in his entire life.

The workflow is as follows

1. Take layers from a previously trained model.
2. Freeze them, so as to avoid destroying any of the information they contain during future training rounds.
3. Add some new, trainable layers on top of the frozen layers. They will learn to turn the old features into predictions on a new dataset.
4. Train the new layers on your dataset.

vgg19 = VGG19(weights='imagenet',include_top=False,input_shape=(128,128,3))

for layer in vgg19.layers:
    layer.trainable = False
    
model = Sequential()
model.add(vgg19)
model.add(Flatten())
model.add(Dense(2,activation='sigmoid'))
model.summary()

include_top=false:

you have used a pre-trained VGG19 model but are not using the last output layer and using your own output layer. We will specify the input size.

layer.trainable = False

We are freezing the weights of all the layers. So now the weights will not be updated for all the layers of the pre-trained model.

model.add(Dense(2,activation='sigmoid'))

Next up we will add our own classifier on top of the pre-trained model. To have a result of our specific classification. Here we specify the number of classes we want to categorize your data. We only want 2 classes, Mask or No Mask. We will use the sigmoid function as our activation function.

Complete Face Mask Detector Code

You can download the train, test, validation data from here and mask image data from here. For the HAAR cascade download the data from here

The green rectangle is due to the distancing. while the red rectangle implies no minimum distancing. More photos classified using the same code.

Conclusion

We have classified our images into masked and non-masked. For that, we used face detection, Image augmentation, Transfer learning, and Euclidean distance to find the distance between people. We can use this model on our streets to identify people not wearing masks or maintaining a minimum distance and fine them accordingly. Police don’t have to manually identify people and urge them to mask up.