Creating multi-exposure keyframe image displays with FFmpeg and ImageMagick

While I was testing visualization of some videos from the AIST database earlier today, I wanted to also create some “keyframe image displays”. This can be seen as a way of doing multi-exposure photography, and should be quite straightforward to do. Still it took me quite some time to figure out exactly how to implement it. It may be that I was searching for the wrong things, but in case anyone else is looking for the same, here is a quick write up.

The current procedure is done using a combination of two very handy command line tools: FFmpeg and ImageMagick. I would like to add it to both the Matlab and Python versions of the Musical Gestures Toolbox as well, but will need to figure that out another time.

In this example I will use a hip-hop dance video from the AIST database:

The first step is to extract keyframes from the video file using this one-liner ffmpeg command:

ffmpeg -skip_frame nokey -i *.mp4 -vsync 0 -r 30 -f image2 t%02d.tiff

This will use the keyframes from the MP4 file, which should be faster than doing a new analysis of the file. It could, of course, also be possible to sample the video at regular intervals, but the keyframes seem to work fine for my usage. I also choose to save the exported keyframes as TIFF files to avoid running multiple rounds of compression on the files. The end result is a bunch of keyframe images that can be used for further processing.

Automagically exported keyframe images.

In my search for a solution, I tried a lot of complex things. But it turned out to be super-simple to get what I wanted:

convert *.tiff -background white -compose darken -flatten keyframes.jpg

Here we use the convert function of ImageMagick to add all the exported keyframes together to one combined image:

Keyframe image display of hip-hop video.

Since the dancer was moving in more or less the same place all the time, it is quite compact. Running the same functions on another video of a contemporary dancer, on the other hand, shows some of the potential of this visualization method. Here is the video:

Which results in this keyframe display image:

Besides being cool to look at, it is also quite informative when it comes to telling what is going on in the video. You get information about the temporal and spatial movement of the dancer, although it is difficult to understand exactly when she was moving where.

Next is to also include these methods in the Musical Gestures Toolbox.

Creating individual image files from presentation slides

How do you create full-screen images from each of the slides of a Google Docs presentation without too much manual work? For the previous blog post on my Munin keynote, I wanted to include some pictures from my 90-slide presentation. There is probably a point and click solution to this problem, but it is even more fun to use some command line tools to help out. These commands have been tested on Ubuntu 19.10, but should probably work on many other systems as well, as long as you have installed pdfseparate and convert.

After exporting a PDF from the Google Presentation, I made a separate PDF file of each slide using this command:

pdfseparate input.pdf output%d.pdf

This creates a bunch of PDF files with a running number. Then I ran this little for loop:

for i in *.pdf; do name=`echo $i | cut -d'.' -f1`; convert -density 200 "$i" "$name.png"; done

And voila, then I had nice PNG files of all my slides. I found that the trick is to use the “-density 200” setting (choose the density that suit your needs), since the default resolution and quality is too low.

Installing Ubuntu on a HP Pavilion laptop

So I decided to install Ubuntu on my daughter’s new laptop, more specifically an HP Pavilion. The choice of this particular laptop was because it looked nice, and had good specs for the money. It was first after the purchase I read all the complaints people have about the weird UEFI implementation on HP laptops. So I started the install process with some worries.

Reading on various forums, people seemed to have been doing all sorts of strange things to be able to install Ubuntu on HP laptops, including modifying the UEFI setup, changing the BIOS, and so on. I recall that on my Lenovo laptop I had to work quite a bit to turn off all the fancy auto-Windows-stuff.

I am not sure if HP has changed something recently or not, but the final procedure was super-easy: I just hit the F9 button on startup and got a regular “old-school” boot selector. Here I chose the USB drive, and the Ubuntu installer fired up.

I have installed Linux (primarily various Ubuntu versions) on several laptops over the years, and it is very seldom that I get into problems with drivers. Also this time, things went smoothly; everything worked perfectly right after the install. I think it is crucial to continue repeating this message because I still hear people saying that it is tricky to get Ubuntu to play with different hardware. True, there used to be driver issues some years ago, but I haven’t experienced that in five years or so.

Which Linux version to choose for a 9-year old?

My 9-year old daughter is getting her first laptop. But which OS should she get started with?

I have been using various versions of Ubuntu as my main OS for around 5 years now, currently using Ubuntu Studio on my main laptop. This distro is based on XFCE, a very lightweight yet versatile OS. The reason for choosing Ubuntu Studio over the regular XUbuntu was to get a bunch of music apps by default. I haven’t been able to explore these as much as I wanted to, unfortunately, primarily due to everything happening at our new centre (RITMO) and master’s programme (MCT).

Even though I like Ubuntu Studio myself, it is not a distro I would install on my daughter’s machine. Buying a new computer with Windows 10 pre-installed, one could argue that it would be best to leave her with that. This may also help her to be more familiar with the computers they are using at school, which run Windows 7 at the moment. But the question in the store about whether I wanted to buy some antivirus-software with the new laptop, was enough to ensure me that a Linux distro would be a better choice.

I have heard that some people like distros such as Edubuntu for kids, but it does not seem to be maintained? After thinking about it for a little while, I have concluded that it is probably useful for a kid to learn to use a normal OS. If you compare how things were a decade or two ago, most modern-day OSes are comparably easy to use anyways.

Finally I decided to make it simple, and installed the regular Ubuntu distro based on GNOME. It looks “modern”, has large icons, and is fairly easy to navigate due to the streamlining of menus, and so on.

Rotate lots of image on Ubuntu

I often find myself with a bunch of images that are not properly rotated. Many cameras write the rotation information to the EXIF header of the image file, but the file itself is not actually rotated. Some photo editors do this automagically when you import the files, but I prefer to copy files manually to my drive.

I therefore have a little one-liner that can rotate all the files in a folder:

find . *.jpg -exec jhead -autorot {} \;

It works recursively, and is very quick!