Particularly notable is the U.S. Department of Transportation's
Intelligent Vehicle Initiative .
Driver drowsiness is one specific form of human error that has been well studied.
Consequently, we attempted to devise a camera + image processor system to take
After trying to use cameras in the visible range of the spectrum to detect
Below are some examples of using an "ordinary" digital camera during
day and nighttime driving conditions. As you can see, not enough light is
captured by the particular camera we were using to handle both
situations. We felt like we needed an imaging system that could handle
both daytime and nighttime conditions. So, using the literature as an
example, we chose an IR camera. The daytime images are especially encouraging, since only the eyes seem to
show up in the picture at all. Neural networks are programming abstractions that attempt to make
decisions based on a complex network. By training this network, the
neural network attempts to classify how close an input is to the
space spanned by the training set. Thresholding in this closeness
measure produces a result. Neural networks are notorious for being
difficult to train. Originally, we had an idea to use some code developed for another
class to perform face detection and modify this code to perform eye
detection. Conveniently, MATLAB has a neural network toolbox which
this implementation utilized. It seemed like a good chance to learn
about neural networks and their applications to image processing.
However, as you will see below, we quickly abandoned this idea for
more conventional image processing techniques. Correlation Methods Template matching is one possible technique for searching for a pattern
within an image. Typically, a suitable "template" is chosen as a feature
to search for within in an image. In our case, we chose an "average" eye
by averaging over test cases. An example of an average is shown
below. Template matching is finding the minimum error between "windows" of the
image and the template. Alternatively, this is searching for a maxima of
the convolution of the image with the template. However, to avoid a bias
towards bright sections of the image, each window should be "mean removed"
to insure proper correlation. Hough Transform The Hough transform is transform that searches for maxima in a parametric
space. Thus, any "shapes" that can be expressed parametrically are well
suited to techniques using the hough transform. In our images, the pupils
of the eye formed nearly perfect circles. If we restrict the distance to
the camera, this also roughly fixed the radius of the pupil at 5 pixels. Once we find the maxima in the parametric space, we perform the inverse
hough transform to determine where the original circles are in the image.
Subsequent processing can occur in these regions to further increase SNR. Our implementation of the Hough transform for circular objects is based on
this code developed at
the University of Minnesota by Dan Pou.
As the eye closure occurrences dramatically increase during the 10-second
Theory and Methodology
Using IR illumination and IR camera
illuminated with IR lightNeural Network
Results and Discussion
Neural Network
We attempted to use a matlab neural network face recognition routine developed by Scott Sanner for CS223B. Our original idea was to modify this routine to detect eyes. Though the previously noted literature by Wierwille points to neural networks for pattern recognition as a promising approach, this routine had severe difficulty actually detecting faces, most likely due to insufficient or improper training. A selected result is shown for the face detection implementation.
It doesn't seem to find a face at all! After playing with this method, we dropped it to develop our own techniques from methods presented in class.
When attempting circle detection with the
Hough Transform, it is important to remember that it the function is dependant
on a black/white edge map (shown on the bottom left of the animation). We used
an edge detection function with varying thresholds to find the pupils with the
least amount of background noise. In the first example, this thresholding had
to be decreased to such a level that it also captured a lot of background edges
in the hair and ears. Unfortunately, when this edge map is passed into the
Hough detection function, it shows that there are many potential circles of a
radius of five pixels in the image (shown on the bottom right of the
animation). This Hough image is then morphologically thresholded (using the
erode and dissolve functions) such that the most likely circles are
distinguished from the noise (top right of animation) and the corresponding
coordinates are plotted to the original image.
