Chapter NUI-14.   Using the Leap Motion Controller

This chapter does not appear in the book.

A few weeks ago, I finally received my much-anticipated Leap Motion controller, a device that grants us mere mortals the ability to interact with a computer by waving our hands and fingers. The picture on the right shows my left hand wafting around over the controller (the small metallic and black glass device on the table).

A slew of user reviews have appeared, including from The New York Times, Engadget, and TechCrunch. I tend to agree with their conclusions that the device is tricky and tiring to use. But it's early days for such an innovative idea, and the technology may be a more natural match for hand-held devices such as smartphones and tablets. It also nicely complements the Kinect sensor, with the Kinect handling whole-body tracking across meters while the Leap is fine-tuned for hand and finger tracking and gestures, accurate to millimeters.

The main thrust of this chapter is how to program with the Leap using its Java API (there are also libraries for Python, C++, C#, Objective C, and JavaScript). Free registration at the Leap Motion developer website allows you to download various goodies such as the Leap SDK, API documentation, technical overviews, and access forums (which are also archived online).

I'll describe three applications in this chapter:

1. A GUI for a slightly modified version of the SDK's Java example. It displays a large amount of rapidly changing data about detected hands, fingers, and gestures. A screenshot of the application, called LeapViewer, is shown below.
    
2. A 2D canvas which illustrates how to use hands, fingers, and the finger twirling gesture to move, draw and delete colored dots. A screenshot of LeapDoodle is included below.
    
3. A hand-controlled 3D model (the robot picture below), which can glide across a checkerboard floor and rotate around its vertical axis. This example is a simplification of a Java 3D application described at length in the online chapter 3D Sprites. I'll focus on the Leap-related code here, and won't be explaining the complexities of Java 3D. The application is called LeapModel3D.
    

Downloads

• The PDF file for the draft chapter (888 KB). Last updated: 7th August 2013.
  
• Zipped code (468 KB). Last updated: 7th August 2013.
  

• Up to the Killer Game Programming in Java Index Page
• Back to NUI Chapter 10. Reading Your Mind With the MindWave
• On to NUI Chapter 15. The Wiimote, the PC, and Java