![[Vertical Waving PIC]](vertWave.jpg)
Kinect Chapter 13.
FAAST-style Body GesturesThis chapter was previously labelled as "NUI Chapter 16.10".
The GorillasTracker application from Chapter 4 illustrates how to program with OpenNI's skeletal tracking, rendering each skeleton as connected colored lines (with an attached gorilla head). The screenshot on the right is a reminder of how things look.
The crucial data structure is a HashMap of skeletons called userSkels, which maps user IDs to skeletons made up of joints. The collection of joints for a given skeleton are represented by another HashMap which pairs joint names (SkeletonJoint values) with their (x, y, z) coordinates (SkeletonJointPosition objects). The userSkels declaration:
private HashMap<Integer,
HashMap<SkeletonJoint, SkeletonJointPosition>> userSkels;
When a new user is detected, a ID-skeleton mapping is added to userSkels, which is updated as the user moves, and deleted when the person leaves the scene.
This data structure is utilized in the GorillasTracker application of Chapter 4 to draw the skeletons. This chapter explores its use for detecting body gestures, such as lifting an arm, bringing both hands together, and waving a hand up and down.
To give credit where it's due, this chapter was inspired by the FAAST library. FAAST, short for Flexible Action and Articulated Skeleton Toolkit, allows a programmer to add full-body control to games and VR applications. It's built on top of OpenNI and NITE on a Windows platform, and makes it very easy to map gestures to keyboard and mouse controls, thereby allowing conventional software to be controlled by gesturing. FAAST also includes network support so skeletal information can be transported between machines. The application in this chapter is much less capable, only calling a print method when a body gesture is detected.
The screenshot at the top of this page shows the user with their right arm lowered, which occurs at the end of a series of up and down movements. The output from the gesture detectors includes the following:
:
RH_DOWN 1 on
RH_DOWN 1 off
RH_FWD 1 on
RH_UP 1 on
RH_UP 1 off
RH_FWD 1 off
RH_DOWN 1 on
RH_FWD 1 on
RH_DOWN 1 off
RH_UP 1 on
RH_UP 1 off
RH_FWD 1 off
RH_DOWN 1 on
VERT_WAVE 1 on
RH_FWD 1 on
RH_DOWN 1 off
:
The RH prefix on most of the messages stands for "Right Hand". A vertical wave gesture is represented by the VERT_WAVE output which appears after several up and down swipes of the right hand.