Mocap workshop in Trondheim

I will participate in a motion capture workshop at the Norwegian University of Science and Technology (NTNU) tomorrow. My contribution will consist of the following:

  • Lecture: Introduction to motion capture (in music analysis and performance)
  • Demo 1: Working with video analysis using the Musical Gestures Toolbox
  • Demo 2: The Xsens MVN BioMech mobile mocap suit
  • Workshop: Analysis and performance with Wii controllers, Phidgets accelerometers and Kinect

Below are various resources.


PDF of the presentation.

Demo 1: Musical Gestures Toolbox

The Musical Gestures Toolbox is a collection of modules and abstractions developed in and for the graphical programming environment Max. The toolbox is currently being developed within the Jamoma open platform for interactive art-based research and performance.

Download: Jamoma + UserLib and Max

The toolbox is probably most useful for people that are already familiar with Max programming. People looking for more easy-to-use solutions can check out some of my standalone applications at the fourMs software page. These applications should work on most versions of OSX, as well as on WinXP. I know that there are various issues with Windows Vista and Windows 7, and will try to get these problems ironed out as soon as possible.

Demo 2: XSens

Xsens MVN BioMech is a mobile mocap suit based on inertial sensors (accelerometer, gyroscopes, magnetometers). It excels over camera-based systems in that it is portable and allows for mobile motion capture. I will show how the system can be used both for analysis and performance:

  • Analysis: mocap recordings will be made with the internal Xsens software, and exported to C3D files that will be imported in Matlab using the Mocap Toolbox.

  • Performance: I will also show how the system can be used in realtime, passing data to Max within which the packets will be parsed and used to control sound in realtime.


During the hands-on workshop, participants will be able to try out the above mentioned tools and systems, as well as:

  • Phidgets USB motion sensors.

  • Nintendo Wii controllers, which allow for wireless inertial motion sensing. Here there are several tools available.

    • OSCulator a general purpose tool for reading data from various human interface devices.
    • WiiDataCapture can record data coming from OSCulator, and format them so that they can easily be imported by the MoCap Toolbox
    • junXion is an application for passing on either OSC or MIDI from Wii controllers and other human interface devices.
  • Microsoft Kinect sensor, which allows for inexpensive “3D” motion capture using depth-cameras.

    • Wiki page describing how to work with Kinect in Max

Demonstration videos on using Phidgets electronic kits

I am using the Phidgets eletronics kits when teaching sound programming, and have now made two small videos demonstrating some basic principles.

First, there is a getting started with Phidgets in Max video in Norwegian:


And I have also made a video demonstrating the Phidgets2MIDI application that I developed earlier this year:


I am planning to make some videos showing some more musically interesting use of the electronics and software.