Hasta La Gadget!

Interacting with your computer by waving your hands may require just a pair of multicolored gloves and a webcam, say two researchers at MIT who have made a breakthrough in gesture-based computing that’s inexpensive and easy to use.

A pair of lycra gloves — with 20 irregularly shaped patches in 10 different colors — held in front of a webcam can generate a unique pattern with every wave of the hand or flex of the finger. That can be matched against a database of gestures and translated into commands for the computer. The gloves can cost just about a dollar to manufacture, say the researchers.

“This gets the 3-D configuration of your hand and your fingers,” says Robert Wang, a graduate student in the computer science and artificial intelligence lab at MIT. “We get how your fingers are flexing.” Wang developed the system with Jovan Popović, an associate professor of electrical engineering and computer science at MIT.

The technology could be used in videogames where gamers could pick up and move objects using hand gestures and by engineers and artists to manipulate 3-D models.

“The concept is very strong,” Francis MacDougall, chief technology officer and co-founder of gesture-recognition company GestureTek, told Wired.com. “If you look at the actual analysis technique they are using it is same as what Microsoft has done with Project Natal for detecting human body position.” MacDougall isn’t involved with MIT’s research project.

MIT has become a hotbed for researchers working in the area of gestural computing. Last year, an MIT researcher showed a wearable gesture interface called the “SixthSense” that recognizes basic hand movements. Another recent breakthrough showed how to turn a LCD screen into a low-cost, 3-D gestural computing system.

The latest idea is surprisingly easy in its premise. The system hinges on the ability to use a differentiated enough pattern so each gesture can be looked up quickly in a database.

For the design of their multicolored gloves, Wang and Popović tried to restrict the number of colors used so the system could reliably distinguish one color from another in different lighting conditions and reduce errors. The arrangement and shapes of the patches were chosen such that the front and back of the hand would be distinct.

Once the webcam captures an image of the glove, a software program crops out the background, so the glove alone is superimposed on a white background.

The program then reduces the resolution of the cropped image to 40 pixels by 40 pixels. It searches through a database that contains 40 x 40 digital models of a hand, clad in the distinctive glove showing different positions. Once match is found, it simply looks up the corresponding hand position.

Since the system doesn’t have to calculate the relative positions of the fingers, palm and back of the hand on the fly, it can be extremely quick, claim the researchers.

And if the video is to be believed, the precision with which the system can gauge gestures including the flexing of individual fingers is impressive.

A challenge, though, is having enough processing power and memory so gestures made by a user can be looked up in a database quickly, says MacDougall.

“It takes hundreds of megabytes of pre-recorded posed images for this to work.,” he says, “though that’s not so heavy in the computing world anymore.”

Another problem could be getting people to wear the gloves. Let’s face it: No one wants to look like Kramer in a fur coat from a episode of Seinfeld or an extra in the musical Joseph and the Technicolor Dreamcoat.

MacDougall says the pattern on the gloves can be tweaked to make them less obvious.

“If you want to make it more attractive, you could hide the patterns in a glove using retro-reflective material,” he says. “That way you could [create] differentiable patterns that wouldn’t be visible to the naked eye but a camera’s eye could see it.”

Wang and Popović aren’t letting issues like fashion dictate their research. They say they are working on a design of similarly patterned shirts.

Photo: Jason Dorfman/CSAIL
Video: Robert Y. Wang/Jovan Popović

See Also:

• Gestural Computing Breakthrough Turns LCD Into a Big Sensor
• Laptop Makers Shy Away From Multitouch Technology
• Gesture Cube Responds To Waving Your Hand
• Google Adds Gesture Search to Android Phones
• Gesture Controlled Mouse Makes Its Retail Debut