Researchers Create Touch-Based Interface That Can Be Applied Anywhere
April 26, 2013

Researchers Create Touch-Based Interface That Can Be Applied Anywhere

Brett Smith for - Your Universe Online

In 2002, Tom Cruise used “smart gloves” to manipulate images and video on a virtual touchscreen in Steven Spielberg´s Minority Report. However, Microsoft´s Kinect system, which debuted in 2010, makes that interface look antiquated.

But now, a team of engineers at Carnegie Mellon University (CMU) have taken a different tack and demonstrated a projector-based system that allows touch-based interfaces to be created anywhere with the swipe of a hand.

For example, the system, dubbed WorldKit, allows a user to draw a functioning remote control on the arm of their sofa or an interactive calendar on an area of wall space.

Using a ceiling-mounted camera and projector, the CMU team developed a way to record room geometries, sense hand gestures and project desired images onto surfaces.

"Depth sensors are getting better and projectors just keep getting smaller," said Robert Xiao, a doctoral student at Carnegie Mellon's Human-Computer Interaction Institute (HCII).

Xiao said the team is working toward a simple, easy-to-install system.

"We envision an interactive 'light bulb' – a miniaturized device that could be screwed into an ordinary light fixture and pointed or moved to wherever an interface is needed,” he said in a statement.

The researchers said their system does not require calibration, as it automatically adjusts its sensing and projection functions to the orientation of a selected surface. Users can create switches, message boards, indicators and a variety of other interface items from a designated menu.

The CMU engineers said they expect their system to allow users whole interfaces by simply using gestures.

"People have talked about creating smart environments, where sensors, displays and computers are interwoven," said doctoral student Chris Harrison, who helped design the system. "But usually, that doesn't amount to much besides mounting a camera up on the ceiling. The room may be smart, but it has no outlet for that smartness.

“With WorldKit, we say forget touchscreens and go straight to projectors, which can make the room truly interactive,” he explained.

While the WorldKit team is currently focused on creating interfaces based on surfaces, the engineers anticipate the technological advances of the very near future will allow them to apply the system to free three-dimensional space. They added higher resolution versions of Kinect-like systems will allow for WorldKit to perceive detailed finger gestures.

"We're only just getting to the point where we're considering the larger questions," Harrison said, noting a variety of applications for the system have yet to be explored.

While some people readily anticipate the next stage of a Minority Report-style interface, not everyone is embracing the touchscreen revolution unleashed by the iPhone´s debut.

In a recent column for The Awl, commercial animator Christian Brown wrote the futuristic interface shown in the movie may work for cinema, but it has trapped designers in “a world of bad interfaces.”

Brown argued the interface popularized by Spielberg´s vision is counter-intuitive and more natural interfaces might not look as cool, but would be more effective. For example, Brown said a keyless entry system for a car shouldn´t involve making a “secret gesture on the car´s window,” instead of “the car automatically unlocking when you open the handle if you have the key in your pocket.”

(Below) Image Caption: Carnegie Mellon University researchers have developed a depth camera/projector system that enables people to create computer interfaces on everyday surface using hand gestures. Credit: Chris Harrison/Carnegie Mellon University