Hand Sign Classifier
A two-class hand-sign image classifier built with transfer learning on a frozen MobileNetV2, trained on a small set of 165 images. It includes a Colab webcam cell that captures a frame and classifies it.
By Jawher Ennaceur
View on GitHub- Transfer Learning
- Image Classification
- Data Augmentation
- TensorFlow And Keras
- Computer Vision
The Problem
The dataset is small (165 images in two classes), which makes training a CNN from scratch a poor fit. This project uses pretrained ImageNet features and trains only a small head to separate the two hand-sign classes.
The Approach
I built the classifier in TensorFlow/Keras. Images are read from a class-per-folder directory with ImageDataGenerator (rescale 1/255, resized to 224x224, batch size 10, validation_split=0.2), which gives 133 training and 32 validation images. Augmentation is rotation up to 10 degrees, 5% width/height shifts, 5% zoom, horizontal and vertical flips, and brightness in the range 0.5 to 1.4. The model is a MobileNetV2 with ImageNet weights, include_top=False and frozen weights, followed by GlobalAveragePooling2D, a Dense(512, ReLU) layer, Dropout(0.2) and a Dense(2, softmax) output. I compiled it with Adam (learning rate 0.00008) and categorical cross-entropy, trained for 20 epochs, and plotted training and validation accuracy and loss. For a live check I wrote a Colab cell that uses the browser webcam to capture a 640x480 frame, saves it as test.jpg, resizes it to 224x224, scales it to [0, 1] and calls model.predict.
Results
- Dataset of 165 images in 2 classes (133 training, 32 validation, split by Keras with validation_split=0.2).
- Model is a frozen MobileNetV2 plus a 512-unit dense layer, dropout 0.2 and a 2-way softmax, with about 656,898 trainable parameters in the head (computed from the layer sizes; the notebook does not print model.summary()).
- Trained for 20 epochs (14 steps per epoch, batch size 10); training accuracy was 0.9925 or 1.0 in every epoch, and training loss fell to 0.0112 by epoch 20 (training metrics).
- Validation accuracy on the 32-image split ranged from 0.9375 to 1.0 across the 20 epochs and was 1.0 at epoch 20 (validation loss 0.0467). The validation images come from the same folder as the training data and went through the same random augmentation, and the split is only 32 images, so this is a sanity check, not a generalization estimate.
- The model was saved as need_help.h5 (legacy HDF5 format) in Colab's temporary storage.
- A Colab cell captured a 640x480 webcam frame, applied the same resize and scaling, and the model returned a softmax output of [0.169, 0.831] for the two classes. The notebook does not print the class names, so I cannot say which sign that corresponds to.