Releasing the Musical Gestures Toolbox for Python

After several years in the making, we finally “released” the Musical Gestures Toolbox for Python at the NordicSMC Conference this week. The toolbox is a collection of modules targeted at researchers working with video recordings.

Below is a short video in which Bálint Laczkó and I briefly describe the toolbox:

About MGT for Python

The Musical Gestures Toolbox for Python includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes.

It is possible to run the toolbox from the terminal:

ipython example
Example of running MGT for Python in a terminal.

Many people would probably prefer to run it in a Jupyter notebook:

Screenshots from the example Jupyter Notebook.

The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

History

This toolbox builds on the Musical Gestures Toolbox for Matlab, which again builds on the Musical Gestures Toolbox for Max. The latest version was primarily developed by Bálint Laczkó, Frida Furmyr, and Marcus Widmer.

Read more

To learn more about Musical Gestures Toolbox for Python, take a look at our paper presented at NordicSMC:

“Flattening” Ricoh Theta 360-degree videos using FFmpeg

Ricoh Theta 360-degree camera.

I am continuing my explorations of the great terminal-based video tool FFmpeg. Now I wanted to see if I could “flatten” a 360-degree video recorded with a Ricoh Theta camera. These cameras contain two fisheye lenses, capturing two 180-degree videos next to each other. This results in video files like the one I show a screenshot of below.

Screenshot from a video recorded with a Ricoh Theta.

These files are not very useful to watch or work with, so we need to somehow “flatten” them into a more meaningful video file. I find it cumbersome to do this in the Ricoh mobile phone app, so I have been looking for a simple solution on my computer.

The FFmpeg developers are working on native support for various 360-degree video files. This is implemented in the filter v360, but since it is not in the stable version of FFmpeg yet, I decided to look for something that works right now. Then I came across this blog post, which shows how to do the flattening based on two so-called PGM files that contain information about how the video should be mapped:

ffmpeg -i ricoh_input.mp4 -i xmap_thetaS_1920x960v3.pgm -i ymap_ThetaS_1920x960v3.pgm -q 0 -lavfi "format=pix_fmts=rgb24,remap" remapped.mp4

The result is a flattened video file, as shown below:

Screenshot from a “flattened” 360-degree video.

As for where to split up the video (it is a continuous 360-degree video, after all), I will have to investigate later.

VideoAnalysis v2.0

I am happy to announce a new version of VideoAnalysis, a standalone application for OSX and Windows for creating visualizations and extract motion features from video files.

The GUI of VideoAnalysis v2.0

VideoAnalysis was developed as a standalone version of the Musical Gestures Toolbox. I began working on the toolbox back in 2004, as a collection of modules for Max/MSP/Jitter. Then some people asked me to make a standalone version with some of the core functionality. This version was primarily developed for music researchers, but is also used for sports, dance, healthcare, architecture, and interaction design.

I have less time for development myself these days, so most of the work on the new release has been made by Bálint Laczkó and Aleksander Tidemann. Thanks!

So for anyone reading this: please try out the new version. And if you have problems and/or find any bugs, please report them in the tracker.

Shell script for compressing PDF files on Ubuntu

ubuntu-logo112Back on OSX one of my favourite small programs was called PDFCompress, which compressed a large PDF file into something more manageable. There are many ways of doing this on Ubuntu as well, but nothing really as smooth as I used to on OX.

Finally I took the time to figure out how I could make a small shell script based on ghostscript. The whole script looks like this:

#!/bin/sh
gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/default -dNOPAUSE -dQUIET -dBATCH -dDetectDuplicateImages -dCompressFonts=true -r150 -sOutputFile="compress_$@" "$@"

and by saving it in the nautilus scripts directory:

 ~/.local/share/nautilus/scripts

It shows up when I right click on a file. For most of the files I have tried so far today (uncompressed PDF files), it compresses the files to at least 1/10th of the original size. Very useful, particularly when I only need screen resolution for files.

Screenshot from 2016-06-29 16-55-49

Simple video editing in Ubuntu

I have been using Ubuntu as my main OS for the past year, but have often relied on my old MacBook for doing various things that I haven’t easily figured out how to do in Linux. One of those things is to trim video files non-destructively. This is quite simple to do in QuickTime, although Apple now forces you to save the file with a QuickTime container (.mov) even though there is still only MPEG-4 compression in the file (h.264).

There are numerous linux video editors available, but most of these offer way too many features and hence the need to re-compress the files. But I have found two solutions that work well.

The first one, ffmpeg, should be obvious, although I hadn’t thought that it could also do trimming. However, I often like GUI software, and I have found that Avidemux can do what I need very easily. Just open a file, add start and stop markers for the section to be trimmed, and click save. As opposed to QuickTime, it also allows for saving directly to MPEG-4 files (.mp4) without recoding the file.

There was only one thing that I had to look up, and that was the need for starting the trim section on a keyframe in the video. This is quite obvious when wanting to avoid re-encoding the file, but unfortunately Avidemux doesn’t help in explaining this but only gives an error message. The trick was to use the >> arrows to jump to the next keyframe, and then the file saved nicely.