Kayaking motion analysis

Like many others, I bought a kayak during the pandemic, and I have had many nice trips in the Oslo fiord over the last year. Working at RITMO, I think a lot about rhythm these days, and the rhythmic nature of kayaking made me curious to investigate the pattern a little more.

Capturing kayaking motion

My spontaneous investigations into kayak motion began with simply recording a short video of myself kayaking. This was done by placing an action camera (a GoPro Hero 8, to be precise) on my life vest. The result looks like this:

In the future, it would be interesting to also test with a proper motion capture system (see this article for an overview of different approaches). However, as they say, the best motion capture system is the one you have at hand, and cameras are by far the easiest one to bring around.

Analysing kayaking motion

For the analysis, I reached for the Musical Gestures Toolbox for Python. It has matured nicely over the last year and is also where we are putting in most new development efforts these days.

The first step of motion analysis is to generate a motion video:

From the motion video, MGT will also create a motiongram:

Motiongram of a kayaking video.

From the motiongram, it is pretty easy to see the regularity of the kayaking strokes. This may be even easier from the videogram:

Videogram of a kayaking video.

We also get information about the centroid and quantity of motion:

Centroid and quantity of motion of the kayaking video.

The quantity of motion can be used for further statistical analysis. But for now, I am more interested in exploring how it is possible to better visualise the rhythmic properties of the video itself. It was already on the list to implement directograms in MGT, and this is even higher on the list now.

The motion average image (generated from the motion video) does not reveal much about the motion.

Motion average image of the kayaking video.

It is generated by calculating the average of all the frames. What is puzzling is the colour artefacts. I wonder whether that is coming from some compression error in the video or a bug somewhere in MGT for Python. I cannot see the same artefacts in the average image:

Average image of the kayaking video.

Analysing the sound of kayaking

The video recording also has sound, so I was curious to see if this could be used for anything. True, kayaking is a quiet activity, so I didn’t have very high hopes. Also, GoPros don’t have particularly good microphones, and they compress the sound a lot. Still, there could be something in the signal. To begin with, the waveform display of the sound does not tell that much:

A waveform of the sound of kayaking.

The spectrogram does not reveal that much either, although it is interesting to see the effects of the sound compression done by the GoPro (the horizontal lines from 5k and upward).

A spectrogram of the sound of kayaking.

Then the tempogram is more interesting.

A tempogram of the sound of kayaking.

It is exciting to see that it estimates the tempo to be 122 BPM, and this resonates with theories about 120 BPM being the average tempo of moderate human activity.

This little investigation into the sound and video of kayaking made me curious about what else can be found from such recordings. In particular, I will continue to explore approaches to analysing the rhythm of audiovisual recordings. It also made me look forward to a new kayaking season!

And we’re off: RITMO Centre for Interdisciplinary Studies in Rhythm, Time, and Motion

I am happy to announce that RITMO Centre for Interdisciplinary Studies in Rhythm, Time, and Motion officially started last week. This is a new centre of excellence funding by the Research Council of Norway.

Even though we have formally taken off, this mainly means that the management group has started to work. Establishing a centre with 50-60 researchers is not done in a few days, so we will more or less spend the coming year to get up to speed. The plan is that the faculty group will begin working together from January, while in parallel recruiting PhD and postdoctoral fellows. We aim at moving into our new spaces and having most of the people in place by August 2018, and that is also when we will have the kick-off party.

At least we now have a small web page up and running, and more content will be added as we move along. Here is a short summary of what we will be working on:

RITMO is an interdisciplinary research centre focused on rhythm as a structuring mechanism for the temporal dimensions of human life.
The research will be highly interdisciplinary, combining methods from musicology, psychology and informatics to study rhythm as a fundamental property that shapes and underpins human cognition, behaviour and culture.

Rhythm is omnipresent in human life, as we walk, talk, dance and play; as we tell stories about our past; and as we predict the future. Rhythm is also central to human biology, from the oscillations of our nervous system to our heartbeats, breathing patterns and longer chronobiological cycles. As such, it is a key aspect of human action and perception that is in complex interplay with the various cultural, biological and mechanical rhythms of the world.

RITMO will undertake research on rhythm in human action and perception, using music, motion and audio-visual media as empirical points of departure. Our core idea is that the human ability to experience the world and our actions as rhythmic, points to a basic cognitive mechanism that is in itself rhythmic in nature. The vision of RITMO is to understand more about this cognitive mechanism, and through this generate ground-breaking knowledge about the ways in which humans structure and understand the temporal dimensions of their life.

The centre is interdisciplinary and will combine perspectives and methods from music and media studies, philosophy and aesthetics, cognitive neuroscience, and informatics, using state-of-the-art technologies for motion capture, neuroimaging, pupillometry and robotics.

RITMO is to reveal the basic cognitive mechanism(s) underlying human rhythm, using music, motion and audiovisual media as empirical points of departure.

The research will be highly interdisciplinary, combining methods from musicology, psychology and informatics to study rhythm as a fundamental property that shapes and underpins human cognition, behaviour and culture.

New Centre of Excellence: RITMO

The new centre directors: Anne Danielsen (years 1–5) and Alexander Refsum Jensenius (years 6-10).

I am happy to announce that the Research Council of Norway has awarded funding to establish RITMO –  Centre of Excellence for Interdisciplinary Studies in Rhythm, Time and Motion. The centre is a collaboration between Departments of Musicology, Psychology and Informatics at University of Oslo.

Project summary

Rhythm is omnipresent in human life, as we walk, talk, dance and play; as we tell stories about our past; and as we predict the future. Rhythm is also central to human biology, from the microoscillations of our nervous system to our heartbeats, breathing patterns and longer chronobiological cycles (or biorhythms). As such, it is a key aspect of human action and perception that is in complex interplay with the various cultural, biological and mechanical rhythms of the world.

The vision behind RITMO is to reveal the basic cognitive mechanism(s) underlying human rhythm, using music, motion and audiovisual media as empirical points of departure. No other interdisciplinary research environment has focused solely on rhythm and its direct and indirect impacts before. Given the fundamental role of rhythm in human life, such an endeavour is long overdue.

RITMO will undertake research on rhythm in human action and perception, and on the aesthetic and cultural ‘texts’ that such processes elicit. This venture will benefit from the combined perspectives of the humanities, cognitive neuroscience, social sciences and informatics. Now is the right time to establish such a centre, because we can finally explore some of the larger questions of the humanities via state-of-the-art technologies for motion capture, neuroimaging, pupillometry and robotics. Such a research strategy is as novel as it is essential to any engagement with the impact of human rhythm. RITMO will generate groundbreaking knowledge about the structuring and understanding of the temporal dimensions of human life. As such, it will change how we view human cognition and supply a cornerstone for the future exploitation of rhythm in applications for well-being and rehabilitation.