Gesture Recognition

Humans have always wanted to interact with their environment. Humans communicate with other people as well as objects daily to get work done. Computers and machines have become important elements in our society. We use them to perform various tasks. Humans still rely on keyboards, mouse or buttons to interact with machines. This has tremendously affected the naturalness of interaction. Humans interact with each other using speech and gestures and it would be great if we can do the same with machines. Recently a lot of research has been done for using speech as input for systems. Interaction using only speech has some drawbacks. The speech recognition system has to be trained by the user and even then they are not accurate enough. Also designing speech input based systems that can be operated by just one user don’t make sense.

Gesture recognition has been used to complement speech recognition. Gesture recognition can be done by interpreting the motion of a human arm. E.g. a hello wave. Gesture recognition was initially done using glove based techniques. These gloves had mechanical sensors to measure the arm and finger joint angles. The user has to wear a mechanical glove and carry the load of wires attached to it. This method of gesture recognition defeats the purpose (naturalness of interaction) itself. Researchers have now tried gesture recognition using visual sensors. Presently common human hand gestures like pointing, wave, and stop as well as American Sign Language gestures can be recognized using vision. Further research is needed to understand more complex human gestures and use both hand tracking for gesture recognition. I am presently working on a gesture recognition system.

PROPOSAL:

We plan to develop a gesture recognition system which permits a human user wearing a specially marked glove to command a computer system to carry out predefined gesture action commands. Here they basically get the initial and the final position of the points of interest (markers) on the glove and extract their motion vector, since different gestures have different sets of motion vectors for the markers they can be discriminated. After solving the static gesture recognition problem we plan to form a library of simple dynamic human gestures which can be recognized and be used for intelligent control of machines. This would require tracking the human hand. Such gestures would consist of discrete abrupt motions and a multi modal particle filter would be needed to track it using a library of motion models. Bandwidth limitation is a major concern in video tele-conferencing. Our system can be used to for video coding: image sequence are described by states (position, scale and orientation) of all objects in the scene and only change in model parameters are sent instead of the whole sequence of images.