Kinect Chapter 15.   Using the Kinect's Microphone Array

This chapter is not included in the book.

The previous chapter was about speech recognition, with audio recorded from the PC's microphone rather than the Kinect. Frankly, a bit of a cheat, but still useful.

This chapter shows how to capture sound from the Kinect's microphone array, so there's no longer any need to feel guilty about using an extra microphone. The trick is to install audio support from Microsoft's Kinect SDK which lets Windows 7 treat the array as a standard multichannel recording device. Care must be taken when mixing this audio driver with OpenNI, but the payoff is that the microphone array becomes visible to Java's sound API. The main drawback is that this only works on Windows 7 since Microsoft's SDK only supports that OS.

My Java examples start with several tools for listing audio sources and their capabilities, such as the PC's microphone and the Kinect array. I'll also describe a simple audio recorder that reads from a specified source and saves the captured sound to a WAV file.

One of the novel features of Microsoft's SDK is support for beamforming – the ability to calculate the direction of an audio source. It's possible to duplicate this using the Java SoundLocalizer application developed by Laurent Calmes.

I finish with another version of my audio-controlled Breakout game, this time employing the Kinect's microphone array instead of the PC's mike.

Downloads

• The PDF file for the draft chapter (792 KB). Last updated: 14th March 2012.
  
• Zipped code (192 KB). Last updated: 14th March 2012.
  IMPORTANT: the installation details for the OpenNI and NITE software have changed since the book was published; please read the details on the main KOPS page.

• Up to the Kinect Open Source Programming Secrets Index Page
• Back to Kinect Chapter 14. Speech Recognition