MGT

Yesterday, I wrote about some reflections I had during Olgerta Asko’s PhD defence. Today, while chopping up the video recording to put on the RITMO web page, I thought that it might help to use a videogram to assist with segmentation. Videograms A videogram is similar to a motiongram, the main difference being that the videogram uses the regular video image as input to the “compression” instead of a motion video. Both give an impression of what is in a video file over time. We have functions for creating both videograms and motiongrams in the Musical Gestures Toolbox, but they are optimised for using other functions in the toolbox. ...

I have been running Python-based Jupyter Notebooks for some time but never thought about using environments before quite recently. I have heard people talking about environments, but I didn’t understand why I would need it. Two days ago, I tried to upgrade to the latest version of the Musical Gestures Toolbox for Python and got stuck in a dependency nightmare. I tried to upgrade one of the packages that choked, but that only led to other packages breaking. I suddenly also found myself in a situation where one package wanted a newer version of a dependency while another wanted an older version. My approach to installing all the packages I would ever need was not a good idea. Then I realized that this is why people use environments. ...

I participated in the conference The Aesthetics of Absence in Music of the Twenty-First Century at the Department of Musicology the last couple of days. Judith Lochhead started her keynote lecture with a clip from the movie The art of flying by Jan van Ijken. This is a beautiful short film based on clips of flocking birds: The art of flying from Jan van IJken on Vimeo. Of course, I wanted to see how some video visualizations would work, so I reached for the Musical Gestures Toolbox for Python. After purchasing and downloading the video, I run a version of ipython, load the toolbox and import the video file: ...

Today, I ran a workshop called Video Visualization together with RITMO research assistant Joachim Poutaraud. The workshop was part of the Digital Scholarship Days 2023 organized by the University of Oslo Library, four days packed of hands-on tutorials of various useful things. Presentation slides made by Jupyter Notebook Joachim has done a fantastic job updating the Wiki with all the new things he has implemented in the toolbox. However, the Wiki is not the best thing to use in a workshop, it has too much information and would create an information overload for the participants. ...

Last year, I wrote about video-based motion analysis of kayaking. Those videos were recorded with a GoPro Hero 8 and I tested some of the video visualization methods of the Musical Gestures Toolbox for Python. This summer I am testing out some 360 cameras for my upcoming AMBIENT project. I thought I should take one of these, a GoPro Max, out for some kayaking in the Oslo fjord. Here are some impressions of the trip (and recording). ...

I came across a fantastic performance of a Bach prelude played on Boomwhackers by Les Objets Volants. https://www.youtube.com/watch?v=Y5seI0eJZCg It is really incredible how they manage to coordinate the sticks and make it into a beautiful performance. Given my interest in the visual aspects of music performance, I reached for the Musical Gestures Toolbox to create some video visualisations. I started with creating an average image of the video: ...

In some recent posts I have explored the creation of motiongrams and average images, multi-exposure displays, and image masks. In this blog post I will explore different ways of generating keyframe displays using the very handy command line tool FFmpeg. As in the previous posts, I will use a contemporary dance video from the AIST Dance Video Database as an example: The first attempt is to create a 3x3 grid image by just sampling frames from the original image. I spent some time exploring different ways of doing this. It is possible to do it with a one-liner: ...

Yesterday I presented the Musical Gestures Toolbox for Matlab in the late-breaking demo session at the ISMIR conference in Paris. The Musical Gestures Toolbox for Matlab (MGT) aims at assisting music researchers with importing, preprocessing, analyzing, and visualizing video, audio, and motion capture data in a coherent manner within Matlab. Most of the concepts in the toolbox are based on the Musical Gestures Toolbox that I first developed for Max more than a decade ago. A lot of the Matlab coding for the new version was done in the master’s thesis by Bo Zhou. ...

Two weeks ago Lars Adde defended his PhD entitled Prediction of cerebral palsy in young infants. Computer based assessment of general movements at NTNU in Trondheim. I have contributed to this research through development of the General Movement Toolbox, a variant of my Musical Gestures Toolbox. This toolbox he has used to analyse video material of children with fidgety movements, with the aim of being able to predict cerebral palsy at an early stage. The dissertation is a milestone in Norwegian physiotherapy research, and I am very happy to have been able to contribute. ...