Steven Spielberg’s 2002 production Minority Report was a pretty good movie. Its artistic worth, however, is nothing compared to its enduring significance as a convenient touchstone for anyone who’s talking about the future of user interfaces.
In the film, of course, Tom Cruise played a policeman who efficiently rifles through vast digital files of information by waving his arms around as if he were conducting a symphony. As new real-world forms of input involving gestures have proliferated, people have instinctively compared all of them to Cruise’s on-screen gesticulations.
But no new means of input has ever felt as Minority Report-like as the Leap Motion controller, an add-on for Windows PCs and Macs. Like Microsoft’s Kinect, the controller, from a San Francisco startup, uses cameras and infrared sensors to track your motion. But instead of watching your whole body, Leap Motion’s gizmo focuses — very precisely — on your hands and fingers. It can tell how many fingers you’ve extended, the angle of your palms and which direction your hands are moving, and at what speed. Basically, if the Minority Report interface were real rather than a splashy special effect, it would work just like this.
After a couple of delays — it was originally supposed to show up early this year — Leap Motion’s $80 gadget is now shipping to folks who pre-ordered it, and is scheduled to arrive in Best Buy stores on July 28. I’ve been trying a review unit supplied by the company with both Windows and a Mac. It’s been an enjoyable experience, though also one which reminded me, in multiple ways, that this whole idea is still a brand-new work in progress.
Barely larger than a generously-proportioned steak fry, the three-inch controller plugs into your PC or Mac via USB and sits in front of your keyboard, where it’s easy to wave your hands within its view. (It can sense motion up to two feet above its surface and two feet to all four sides.) The package includes a long cable, useful for connections to a port on the back of a desktop computer, and a shorter one that’s handy for laptops.
When I heard about the controller, I assumed that it would require calibration of some sort and would need to be positioned at some precise location in between me and my computer screen. Nope. Download and install Leap Motion’s software, plug in the controller and you’re ready to go. As long as the controller is on a flat surface, it works: I even kicked back in a beanbag chair, balanced it on my MacBook Air’s touchpad and used it in my lap.
So what can you do with this little technological wonder? At launch, there are almost 80 answers to that question, in the form of apps which support Leap Motion’s newfangled approach to input. You download most of them — the cost ranges from nothing to $4.99 — from a specialized, browser-based app marketplace called the Airspace Store. (Developers can choose to support Windows, Mac or both; Airspace, oddly, shows you all apps regardless of whether they’ll run on your computer.)
The launch apps come from an array of companies and cover a lot of ground: creative tools, education, games, music, utilities and more. Some are Leap-Motionized versions of well-known existing wares such as Google Earth and Cut the Rope. There’s a New York Times app, a car-shopping one from AutoTrader.com and Corel Painter Freestyle, a spinoff of a venerable natural-media art program. Still other programs are all-new, designed with the controller in mind.
Games are a natural for Leap Motion. Without the controller, it’s tough to imagine one called Dropchord existing: Both retro and futuristic, and addictive fun, it lets you point at the screen with one or more fingers on both hands to zap targets and avoid hazards. Music is another obvious application: Various apps let you play tunes on virtual pianos, harps and drum sets without touching anything, a feat formerly possible only with a theremin.
Figuring out how to manipulate a cursor or other element by moving your hand up, down, left and right is generally a cakewalk. But some apps pay careful attention to how close your hand is to the screen, divvying up the space above and around the controller into zones and doing different things depending on which zone your hand is in. It’s hard at times to know which one you’re in until you accidentally leave it. Gesturing to press an on-screen icon or other element is also challenging: I tended to miss upon first attempt. And sometimes second and third attempts.
Consequently, the more ambitious a Leap Motion app’s capabilities, the less elegantly intuitive it tends to feel. A program called Touchless, available in Windows and Mac variants, lets you use the controller to do operating system stuff you might otherwise perform via a mouse or touchpad, such as clicking icons and scrolling through documents such as web pages. I found the process unwieldy, especially on a Mac. When I visited Leap Motion’s office, one of the company’s staffers — a whiz in most of the apps he demoed — jabbed repeatedly at a Windows 8 icon before it registered his input; somehow, that made me feel better.
With practice, all this should get easier. (Hey, getting comfortable with a QWERTY keyboard or a mouse requires time, too.) But there’s another issue which the Leap Motion controller has which keyboards and mice don’t: It’s so new that there’s not much consistency about how it works in different applications.
In the New York Times app, for instance, you scroll downwards by twirling your finger clockwise and upwards by twirling counter-clockwise. It’s easy — and even entertaining. Using Touchless, however, you scroll using an entirely different sort of gesture which feels a bit like pawing at the screen; I had more trouble getting the hang of it.
Another example: There are two programs which let you paint by pointing, Deco Sketch and Corel’s Painter Freestyle. Both let you do so by extending your fingertip deep into the controller’s range of view, as if you were touching brush to canvas. But their other features — such as those for choosing different art tools — are triggered by entirely different gestures.
Now, such inconsistency isn’t evidence that Leap Motion is snoozing at the wheel. Michael Buckwald, the company’s cofounder and CEO, told me that rather than telling the first wave of developers precisely how to use the controller in their applications, it’s encouraging them to try different approaches. Over time, he says, the developer community will identify the techniques which work best, whereupon they’ll be more predictable from app to app.
Couple that with the tendency of software for radically new hardware platforms to mature over time — as seen with the Kinect and Nintendo’s Wii, among many other examples — and you don’t need to be a technological Pollyanna to conclude that better Leap Motion apps will probably arrive in the coming months.
The current plug-in controller might just be the start. Leap Motion is working with HP to build controllers into future PCs, and says that its long-term plans include versions of the technology for smartphones and tablets. Maybe even automobiles. Eventually, it could become the essential, everyday means of input which its creators envision.
For now, the controller is great for games and intriguing for other applications. And at $80, it isn’t a budget-busting luxury. Though not yet all that useful for most folks, it’s already a neat toy — and that’s, well, neat.