Tag Archives: motion

Definitions: Motion, Action, Gesture

I have been discussing definitions of the terms motion/movement, action and gesture several times before on this blog (for example here and here). Here is a summary of my current take on these three concepts:

Motion: displacement of an object in space over time. This object could be a hand, a foot, a mobile phone, a rod, whatever. Motion is an objective entity, and can be recorded with a motion capture system. A motion capture system could be anything from a simple slider (1-dimensional), to a mouse (2-dimensional), to a camera-based tracking system ((3-dimensional)  or an inertial system (6-dimensional: 3D position and 3D orientation). I have previously also discussed the difference between motion and movement. Since motion is a continuous phenomenon, it does not make sense to talk about it in plural form: “motions”. Then it makes more sense to talk about one or more motion sequences, but most probably it makes even more sense to talk about individual actions.

Action: a goal-directed motion (or force) sequence, for example picking up a stone from the ground, playing a piano tone. Actions may have a clear beginning and end, but they may also overlap due to coarticulation, such as when playing a series of tones on the piano. This uncertainty as to how actions should be segmented (or chunked), is what makes them subjective entities. As such, I do not think it is possible to measure an action directly, since there is no objective measure for when an action begins or ends, or how it is organised in relation to other actions. But, based on knowledge about human cognition, it is possible to create systems that can estimate various action features based on measurements of motion.

Gesture: the meaning being expressed through an action or motion. A gesture is not the same as action or motion, although it is related to both of them. As such, a gesture can be seen as a semiotic sign, in which the meaning is conveyed through an action, but it is highly subjective and dependent on the cultural context in which the action is carried out. Also, the same meaning can be conveyed through different types of physical actions. For example, the meaning you convey when you wave “good-bye” to someone may be independent of whether you do it with the left or the right arm, the size of the action, etc.

Unfortunately, with the popularity of motion and gesture studies over the last years, I see that many people use the term gesture more or less synonymously to action or motion. This is particularly the case in the field of “gesture recognition” in various versions of human-computer interaction (HCI).  I think it is unfortunate because we loose the precision with which we can describe the three different phenomena. If we track continuous motion in time and space, it is “motion tracking”. If we aim at recognising certain physical patterns in time and space, I would call it “action recognition” unless we are looking for some meanings attached to the actions. “Gesture recognition” I would only use if we actually recognise the meaning attached to some actions or motion. An example here would be to recognise the emotional quality of the performance  of a violinist. That, however, is something very different than tracking the bowing style.

movement-action
An illustration of my definition of the difference between motion and action

Hi-speed guitar recording

I was in Hamburg last week, teaching at the International Summer Shool in Systematic Musicology (ISSSM). While there, I was able to test a newly acquired high-speed video camera (Phantom V711) at the Department of Musicology.

The beautiful building of the Department of Musicology in Hamburg
The beautiful building of the Department of Musicology in Hamburg
They have some really cool drawings in the ceiling at the entrance of the Department of Musicology in Hamburg.

Master student Niko Plath was friendly enough to set aside some time to set up the system and do a test recording. Niko has been doing some fascinating work on measuring the motion of individual piano strings using the high-speed camera. For this type of study a camera-based approach makes it possible to measure the vibrations of individual strings without having to attach anything to the string or to the board.

Niko Plath setting up the high-speed camera system.

While Niko has recorded the piano strings with a very high speed (500 KHz!) and low resolution(124 x 8 pixels), I was interested in seeing how the camera worked at the maximum resolution (1280 x 800 pixels). At this resolution, the maximum speed is 7 500 frames per second, and the maximum recording duration is 1.1 second.

Even though the recording is short, the processing and exporting of the file (21GB) takes quite some time. So I only had time to make one recording to try things out: a single strumming of all the (open) strings on a guitar, filming the vibrating strings over the sound board.

The setup used for the recording: guitar, two LED lamps and the high-speed camera.

This was just a quick test, so there are several minor problems with the recording: one being that the guitar was placed upside down, so that the lower strings are at the bottom in the recording. Also, I did not hit the upper string very well, so that one only resonates a little in the beginning and decays quickly. Still, there is nothing as beautiful as watching high-speed recordings in slow-motion. Here you can see a version of the recording being played back at 100 frames per second:

Of course, I was interested in creating a motiongram of the recording. Rather than making this using a regular average technique, I rather used a slit-scan approach, selecting a single pixel column in the middle of the soundhole on the guitar. This was done using a few Jamoma modules in Max, and the patch looked like this:

The Max patch used to generate the motiongram from the high-speed video recording.

The full motiongram is available here (TIFF 11 069 x 800 pixels), and below is a JPEG version in a more screen-friendly format. Even though the recording is only a little more than one second long, it is still possible to see the decay of the vibration of the strings, particularly the first strings (from above).

Motiongram of the entire high-speed guitar recording.

Below is a version showing only the beginning of the motiongram, and how the individual strings were strummed. Notice the difference in the “cut-off” of the shape of the wave of each of the strings.

The first 1000 frames of the recording, showing how the strings were strummed.

No new scientific insights here, but it is always fun to see periodic motion with the naked eye. It is a good reminder that auditory and visual phenomena (and the perception of them) are related. Thanks to Niko for helping out, and to his supervisor Rolf Bader for letting me try the system.

Motionlessness

Yesterday Miles Phillips suggested that the word “motionlessness” may be what I am after when it comes to describing the act of standing still. He further pointed me to a web site with a list of the world records for motionlessness. The rules to compete in motionlessness is as follows:

  1. The record is for continuously standing motionless.
  2. You must stand: sitting is not allowed.
  3. No facial movements are allowed other then the involuntary blinking of the eye.
  4. Deep breathing is permitted provided it does not involve observable movement notably greater than that in normal breathing.
  5. No rest breaks are allowed at any point during the event.
  6. The venue for such an event should be such that the general public can view.

But from my point of view, being interested in micromovements, I would be very curious to see how still these record holders actually were.

At the ArtsIT conference next month I will present the results of a study on standstill that I have conducted together with Kari Anne Bjerkestrand. I have given a sneak peek of the data earlier, and below is another figure with plots of motion capture data from the study. The plots show data of a marker placed on the neck, from six different 10-minute long standstill recordings of myself and Kari Anne. It is only the vertical position of the marker that is plotted.

Arj c7 selected5

From the plots we can see that the running marker displacement was at the scale of only a few millimeters, with a maximum displacement of less than 10mm. It can be argued that this is not much, but it certainly is not absolutely still.

One thing is the quantitative data, another is our subjective experience of standing still. Even though we tried our best to stand physically still, we could easily notice how we were swaying back and forth, doing postural adjustments, etc. Observing the video recordings of ourselves afterwards, it is also possible to see these micromovements through visual inspection only.

Based on these findings, I would be very curious to see how still a person can actually stand, not only measured in hours and minutes, but also in millimeters. So to any aspiring world record breakers: please come and do your next attempt in our lab!

Audio recordings as motion capture

I spend a lot of time walking around the city with my daughter these days, and have been wondering how much I move and how the movement is distributed over time. To answer these questions, and to try out a method for easy and cheap motion capture, I decided to record today’s walk to the playground.

I could probably have recorded the accelerometer data in my phone, but I wanted to try an even more low-tech solution: an audio recorder.

While cleaning up some old electronics boxes the other day I found an old Creative ZEN Nano MP3 player. I had totally forgotten about the thing, and I cannot even remember ever using it. But when I found it I remembered that it actually has a built-in microphone and audio recording functionality. The recording quality is horrible, but that doesn’t really matter for what I want to use it for. The good thing is that it can record for hours on the 1GB built-in memory, using some odd compressed audio format (DVI ADPCM).

Since I am mainly interested in recording motion, I decided to put it in my sock and see if that would be a good solution for recording the motion of my foot. I imagined that the sound of my footsteps would be sufficiently loud that they would be easily detected. This is a fairly reduced recording of all my motion, but I was interested in seeing if it was relevant at all.

The result: a 35 MB audio file with 2,5 hours of foot sounds! In case you are interested, here is a 2-minute sample of regular walking. While it is possible to hear a little bit of environmental sounds, the foot steps are very loud and clear.

Now, what can you do with a file like this? To get the file useable for analysis, I started by converting it to a standard AIFF file using Perian in QuickTime 7. After that I loaded it into Matlab using the wonderful MIRToolbox, resampling it to 100 Hz (from 8kHz). It can probably be resampled at an even lower sampling late for this type of data, but I will look more into that later.

The waveform of the 2,5 hour recording looks like this, and reveals some of the structure:

But calculating the smoothed envelope of the curve gives a clearer representation of the motion:

Here we can clearly identify some of the structure of what I (or at least my right foot) was doing for those 2,5 hours. Not bad at all, and definitely relevant for macro-level motion capture.

Based on the findings of a 2 Hz motion peak in the data reported my MacDougall and Moore, I was curious to see if I could find the same in my data. Taking the FFT of the signal gives this overall spectrum:

Clearly, my foot motion shows the strongest peaks at 4 and 5 Hz. I will have to dive into the material a bit more to understand more about these numbers.

The conclusion so far, though, is that this approach may actually be a quite good, cheap and easy method for recording long-term movement data. And with 8kHz sampling rate, this method may also allow for studying micro-movement in more detail. More about that later.

Difference between the terms movement and motion

Terminology is always challenging. I have previously written about definitions of actions and gesture several times (e.g. here,  here, and here) and chapter 2 in the book Musical gestures: sound, movement, and meaning (Routledge, 2010):

 

There are, however, two words/terms that I still find very challenging to define properly and to differentiate: movement and motion. In Norwegian we only have one word (bevegelse) for describing movement/motion, which makes everything much simpler. But when writing in English, which word should be used? and what is the difference?

It only adds to the confusion that Wiktionary defines movement as “physical motion between points in space”. And Wikipedia has a page on motion (in physics), while none of the many movement pages are related to body movement.

During the last years I have asked many native English speakers about the difference between motion and movement, but have not received any good explanations yet. Many of them think they are slightly different, although this is usually based on their feeling rather than on a proper explanation of the difference. Some native speakers think the two words are the same and can be used interchangeably.

I have also asked researchers working on various types of movement-oriented disciplines about their use of the words, and they often tend to stick to one or the other. From these discussions I have come to think that people working in biomechanics and physics prefer motion, while people  working in physiotherapy, dance and music prefer movement. That motion is a more scientific term is is also suggested here. From this we could assume that motion is related to measurable displacement of objects, which the term motion capture attest to, while movement refers to the qualities or meaning of the displacement.

The above assumptions are, however, only my assumptions. So I thought it would be interesting to see if I could get some more empirical data on the topic. So I decided to use the powers of Google to quantify the differences. Here are some figures from google and google scholar:

search term Google Google Scholar
movement 562 000 000 4 120 000
motion 144 000 000 2 210 000
“body movement” 4,830 000 83 000
“body motion” 1 370 000 76 300

So, clearly, movement seems to be used much more frequently than motion in general language, and also in the scientific literature. However, body movement and body motion are used almost the same amount of times in scientific papers.

But what if we search for the use of the two terms in different fields? Then we get these numbers:

search term 1 search term 2 Google Google Scholar search term 2 Google Google Scholar
music +movement 565 000 000 1 960 000 +motion 213 000 000 1 110 000
physics +movement 136 000 000 1 940 000 +motion 64,100 000 1 340 000
mechanics +movement 36 400 000 1 270 000 +motion 46 800 000 1 140 000
biomechanics +movement 6 110 000 163 000 +motion 3 060 000 167 000
physiotherapy +movement 4 580 000 71 200 +motion 2 530 000 38 600
kinesiology +movement 1 690 000 28 900 +motion 1 050 000 20 100

Again, we see that movement is generally used more than motion, even in physics and mechanics. I am quite surprised that music+motion is used so frequently, particularly since movement has a double meaning in music (i.e. parts of a piece).

What to conclude from all of this? I still do not know what the difference between movement and motion is, and the numbers show that movement is used more than motion also in the disciplines that I thought used motion almost exclusively. Still I like the idea that motion is used to describe physical properties, while movement is used to describe the qualities of motion. So I will stick to that for a while myself.

What do you think? Any comments or suggestions are highly welcome!