I am often dealing with folders with lots of files with weird file names. Spaces, capital letters, and so on, often cause problems. Instead of manually fixing such file names, here is a quick one-liner (found here) that can be run in the terminal (at least on Ubuntu) to solve the problem:
rename 'tr/ A-Z/-a-z/' -- *
It is based on a simple regular expression, replacing any spaces with hyphens, and changing any capital letters to lower case.
I take a lot of timelapse shots with a GoPro camera. Usually, I do this with the camera’s photo setting instead of the video setting. That is because I find it easier to delete unwanted pictures from the series that way. It also simplifies selecting individual photos when I want that. But then I need a way to create a timelapse video from the photos easily.
I am continuing my explorations of the great terminal-based video tool FFmpeg. Now I wanted to see if I could “flatten” a 360-degree video recorded with a Ricoh Theta camera. These cameras contain two fisheye lenses, capturing two 180-degree videos next to each other. This results in video files like the one I show a screenshot of below.
Screenshot from a video recorded with a Ricoh Theta.
These files are not very useful to watch or work with, so we need to somehow “flatten” them into a more meaningful video file. I find it cumbersome to do this in the Ricoh mobile phone app, so I have been looking for a simple solution on my computer.
The FFmpeg developers are working on native support for various 360-degree video files. This is implemented in the filter v360, but since it is not in the stable version of FFmpeg yet, I decided to look for something that works right now. Then I came across this blog post, which shows how to do the flattening based on two so-called PGM files that contain information about how the video should be mapped:
As part of my exploration in creating multi-exposure keyframe image displays with FFmpeg and ImageMagick, I tried out a number of things that did not help solve the initial problem but still could be interesting for other things. Most interesting was the automagic creation of image masks from a video file.
This will use the keyframes from the MP4 file, which should be faster than doing a new analysis of the file. It could, of course, also be possible to sample the video at regular intervals, but the keyframes seem to work fine for my usage. I also choose to save the exported keyframes as TIFF files to avoid running multiple rounds of compression on the files. The end result is a bunch of keyframe images that can be used for further processing.
Here we are lucky, because the first frame actually contains the background of the scene. So we can use that frame to create a “foreground” image by subtracting the background image like this:
for i in *.tiff;
do
name=`echo $i | cut -d'.' -f1`;
convert t01.tiff $i -compose difference -composite -threshold 5% -blur 0x3 -threshold 20% -blur 0x3 "$name-mask.tiff"
convert $i "$name-mask.tiff" -compose multiply -flatten "$name-clean.jpg"
done
The end result is a series with the foreground masks:
And then the final result is a series of images in which only the foreground is shown. The “glow” around the images is because of the blur effect used when creating the mask:
Adaptive background
There may also be cases in which there is no readily available background image as we used above, such as in this hip-hop AIST dance video:
Then it is possible to create a background image by averaging over all the images, and hope that this could “remove” the foreground. Here is a one-liner that does this (assuming that you have exported the individual keyframes as mentioned in the beginning of this post):
convert *.tiff -background black -compose lighten -flatten background.tiff
This works quite well, although we can see that the camera right behind the dancer is a little more faint the two others:
Background image created by averaging over all the keyframes.
This background image can then be used to subtract from the other images like we did above:
for i in *.tiff;
do
name=`echo $i | cut -d'.' -f1`;
convert background.tiff $i -compose difference -composite -threshold 5% -blur 0x3 -threshold 20% -blur 0x3 "$name-mask.tiff"
convert $i "$name-mask.tiff" -compose multiply -flatten "$name-clean.jpg"
done
It works very well, except for that the camera behind the performer (that wasn’t masked properly) also shows up in the masked foreground images:
Foreground images created by subtracting an average of the backgrounds.
This method works quite well and has the benefit of being very fast. It is possible to get a better result by creating an average image from the entire video (and not only the keyframes), but this would also take very much longer.
While I was testing visualization of some videos from the AIST database earlier today, I wanted to also create some “keyframe image displays”. This can be seen as a way of doing multi-exposure photography, and should be quite straightforward to do. Still it took me quite some time to figure out exactly how to implement it. It may be that I was searching for the wrong things, but in case anyone else is looking for the same, here is a quick write up.
The current procedure is done using a combination of two very handy command line tools: FFmpeg and ImageMagick. I would like to add it to both the Matlab and Python versions of the Musical Gestures Toolbox as well, but will need to figure that out another time.
In this example I will use a hip-hop dance video from the AIST database:
The first step is to extract keyframes from the video file using this one-liner ffmpeg command:
This will use the keyframes from the MP4 file, which should be faster than doing a new analysis of the file. It could, of course, also be possible to sample the video at regular intervals, but the keyframes seem to work fine for my usage. I also choose to save the exported keyframes as TIFF files to avoid running multiple rounds of compression on the files. The end result is a bunch of keyframe images that can be used for further processing.
Automagically exported keyframe images.
In my search for a solution, I tried a lot of complex things. But it turned out to be super-simple to get what I wanted:
convert *.tiff -background white -compose darken -flatten keyframes.jpg
Here we use the convert function of ImageMagick to add all the exported keyframes together to one combined image:
Keyframe image display of hip-hop video.
Since the dancer was moving in more or less the same place all the time, it is quite compact. Running the same functions on another video of a contemporary dancer, on the other hand, shows some of the potential of this visualization method. Here is the video:
Which results in this keyframe display image:
Besides being cool to look at, it is also quite informative when it comes to telling what is going on in the video. You get information about the temporal and spatial movement of the dancer, although it is difficult to understand exactly when she was moving where.
Next is to also include these methods in the Musical Gestures Toolbox.