Finger Recognition EECS 451 Project

Finger Recognition Project: Introduction and Objectives, In Retrospect

Introduction

The goal of our project is to correctly identify hands in photographic and video inputs. Various photographs will be fed into the program and the output will be a decision of whether there is a individual hand or not inside the image. 

Overview of Methods of Approach

Admittedly, the structuring and understanding of the overall procedure to achieve our goals was unclear initially. Confusion over what constituted methods, steps of methods (pre-processing), and marginally related items accentuated confusion in the initial stages. The plethora of resources beyond our level of knowledge was not helpful either. of other  Throughout the project several approaches were attempted, with varying degrees of success. Below are the various methods attempted:

Image Pre-Processing

Much of our hand samples were taken in less than ideal conditions, and as such the conditions must be improved before further progress can be made.

Image Segmentation Based Approaches

The initial approach used for hand identification was using image segmentation followed by post processing followed by biometrics identification methods. Various criteria were used to segment the image, grey to black and white thresholds, texture filters, and background subtraction methods being most prominent. 

Computer Learning Based Methods

We attempted using Cascade Object Detector to train an object detector to detect hand and also attempted using Foreground Detector to detect foreground and subtract the background.

To train object detector, we need to feed in several positive images and the region that contains the object of interest and also feed in negative image that doesn't contain the object of interests along with some other values. The detector trains in stages and in each stages the detector will try detection on positive and negative images to try to discard any missed classified positive images and discarding true negative images.
trainCascadeObjectDetector('thumbleft5Detector.xml', sp, 'neg5', 'FalseAlarmRate', 0.01, 'TruePositiveRate', 0.8, 'NumCascadeStages', 6);

ForegoundDetector is build-in tool in Matlab that automatically learn the video and extract the foreground by comparing pixels of each frame. I can directly create detector object and use right away.

foregroundDetector = vision.ForegroundDetector('NumGaussians', 3, ...
    'NumTrainingFrames', 8, 'LearningRate', 0.01);

foreground = step(foregroundDetector, frame);

Both of these methods are not very reliable.

Convex Hull

Image Segmentation Based Methods

  -Grayscale to Black and White (Viable only under ideal conditions, hand contrasts heavily with background color)

  -Texture based (Quite successful for images where the background is a distinctive texture from skin)

  -RGB and YCbCr based (Quite successful, the one we ended up using most)

Background Subtraction 

We used Matlab's foreground Detector to try to extract the foreground. However it doesn't extract well due to camera adjusting lighting and exposure. So we tried finding average of the background frames at the beginning of the video until there's lot of changes in the video. Then we do frame-background+background-frame to be my foreground frame and show part on the original frame where the foreground value is greater than a threshold. This method seems promising. If add more filter then this method could give good result.

Object Detection

Matlab's Cascade Object Detector was to train an object detector with several positive and negative images to detect hand. However the detector can't detect hand well and will probably need much more training images to be feasible.

Conclusions

All in all, our project resulted in more successful methods of hand detection than unsuccessful methods. We were able to detect hands and the number of fingers associated with the hand raised with good success, however, there are caveats. Noises of the images are the big problem of our algorithm. We are trying much of the time methods had to selected by a user, and parameters such as what parameters to use with our dilation in image post-processing made it clear that identifying a hand and the number of fingers associated with it is not a trivial task for a computer. In our final algorithm, we take advantage of the value of pixels of the skins and come up with the code that could be able to detect skins well. It works better than the edge detection method that could not detect noisy image. The only disadvantage of the code is when the background is yellow which could sometimes cause an error in counting. For  Two methods we proposed, circular detection and arc intersection, could easily detect the number of fingers showing if the image is clear and not noisy. Even today in programs such as photoshop, in order to segment a object from the image, it often requires human intervention to guide the program as to work perfectly (See below that the . One obvious application relating to this algorithm is the use of this method with gesture recognition. With video input from camera and then background removal, the algorithm could then properly extract the hands out of the background. 

Final checklist of methods used:

1. Grayscale to Black and White (Viable only under ideal conditions, hand contrasts heavily with background color)

2. Texture based, RGB and YCbCr based image segmentation
3. Image Postprocessing and Hand Biometrics

   3.1 Circle finder
     3.1.1 Circular Hough transform

   3.2 Arc Intersection finder

Methods attempted:
1. Cascade Object Detector

2. Foreground Detector
3. Blob analysis
4. Hough transform
5. detectFASTFeatures

Contributions