I have been doing several long recordings with GoPro cameras recently. The cameras automatically split the recordings into 4GB files, which leaves me with a myriad of files to work with. I have therefore made a script to help with the pre-processing of the files.
This is somewhat similar to the script I made to convert MXF files to MP4, but with better handling of the temp file for storing information about the files to merge:
Save the script above as mergevideos.sh, put it in the folder of your files, make it executable, with a command like:
chmod u+x mergevideos.sh
run the file:
and watch the magic.
The script above can be remixed in various ways. For example, if you want a smaller output file (the original GoPro files are quite large), you can use FFmpeg’s default MP4 compression settings by removing the “-c copy” part in the last line above. That will also make the script take much longer, since it will recompress the output file.
I am recording a lot of short videos these days for my sound actions project. Sometimes the recordings end up being rotated, which is based on the orientation sensor (probably the gyroscope) of my mobile phone. This rotation is not part of the recorded video data, it is just information written into the header of the MPEG file. That also means that it is possible to change the rotation without recoding the file. It is possible to see the rotation by looking at the metadata of a file:
ffmpeg -i filename.mp4
Then you will see a lot of information about the file. A bit down in the list is information about the rotation:
Side data: displaymatrix: rotation of -90.00 degrees
Fixing it is as simple as running this command on the file:
To simplify making these wrap-up videos, I am this time around recording them with my Samsung Galaxy s21 Ultra and a set of Røde Wireless GO II microphones. Time is limited when making these videos, so I have decided to quickly trim the files with FFmpeg instead of spending time in video editing software.
I have started shooting videos in 4K, not necessarily because I need it right now, but all my equipment supports 4K these days, and it feels more future-proof. However, FutureLearn does not like 4K and is rather picky about the files to be uploaded:
File format: .mp4 / .mov / .m4v
File size: up to 5GB
Frame rate: 25 fps
Bit rate: min 2 Mbps constant bit rate
Sound: AAC 44khz stereo
So how do you go about creating such files? Well, FFmpeg comes to the rescue again:
The one-liner is relatively self-explanatory. First, I apply a video filter that scales down the video to 1080p and reduces the framerate to 25fps. Then I specify that the audio should be reduced to 44100 Hz. FutureLearn wants a bitrate of 2 Mbps but does not specify a preferred bitrate. I decided to go for 8 Mbps, the suggested bitrate for 1080p uploads to YouTube. I added a minimum bitrate of 2 Mbps at the end, but I don’t think it is necessary since the bitrate used for MP4 files is constant.
All in all, this means that I can do the complete video editing with two simple one-liners, one for trimming the file and the one above for converting to the correct format. That way, I should manage to create two such wrap-up videos each week for the coming weeks.
I often want to create motion videos, that is, videos that only show what changed between frames. Such videos are nice to look at, and so-called “frame differencing” is also the start point for many computer vision algorithms.
We have made several tools for creating motion videos (and more) at the University of Oslo: the standalone VideoAnalysis app (Win/Mac) and the different versions of the Musical Gestures Toolbox. These are all great tools, but sometimes it would be nice also to create motion videos in the terminal using FFmpeg.
I have previously written about the tblend function in FFMPEG, which I thought would be a good starting point. However, it turned out to be slightly more challenging to do than I had expected. Hence, this blog post is to help others looking to do the same.
It does the frame differencing, but I end up with a green image:
I spent quite some time looking for a solution. Several people report a similar problem, but there are few answers. Finally, I found this explanation suggesting that the source video is in YUV while the filter expects RGB. To get the correct result, we need to add a format=gbrp to the filter chain:
The starting point was a bunch of recordings from our recent MusicLab Copenhagen featuring the amazing Danish String Quartet. A team of RITMO researchers went to Copenhagen and captured the quartet in both rehearsal and performance. We have data and media from motion capture, eye tracking, physiological sensing, audio, video, and more. The plan is to make it all available on OSF.
When it comes to video, we have many different recordings, ranging from small GoPro cameras hanging around the space to professional streaming cameras operated by a camera crew. In addition, we have one recording from a Garmin VIRB 360 camera hanging in the chandelier close to the musicians. Those recordings are what I will explore in this post.
There are some obvious problems with this recording. First, the recording is upside down since the camera was hanging upside down from a chandelier above the musicians. The panning and tilting of the camera are also slightly off concerning the placement of the musicians. So it is necessary to do some pre-processing before analysing the files.
Most 360-degree cameras come with software for adjusting the image. The Garmin app can do it, but I already have all the files on a computer. It could also be done in video editing software, although I haven’t explored that. In any case, I look for an option that allows me to batch process a bunch of videos (yes, we have hours of recordings, and they are split up into different files).
Since working on the Ricoh files last year, I have learned that FFmpeg’s new 360 filter is part of the regular release. So I wanted to give it a spin. Along the way, I learned more about different image projections types that I will outline in the following.
The starting point was the equirectangular projection coming out of the Garmin VIRB. The first thing to make it more useful is to flip the video around and place the musicians in the centre of the image.
The different functions of the v360 filter in FFmpeg are documented but not explained very well. So it took me quite some time to figure out how to make the adjustments. This is the one-liner I ended up with to create the image above:
There are some tricks I had to figure out to make this work. First, I use the v360 filter with equirectangular (shortened to e) as both the input and output of the filter. The rotation was done using both the v_flip and h_flip commands, which rotate around both the horizontal and vertical axes. In the original image, the cellist was on the edge. So I also had to turn the whole image horizontally using yaw and move the entire image down a bit using pitch. It took me some manual testing to figure out the correct numbers here.
Since the analysis will be focused on the musicians, I have also cropped the image using the general crop filter (note that you can add multiple filters with a comma in FFmpeg if you try to add another filter, only the last one will be used):
Here I used the flat output type in FFmpeg and did the same flipping, panning and tilting as above. I had to use slightly different numbers for yaw and pitch to make it work, though. Also, here I added some cropping to focus on the musicians:
The equi-angular cubemap should have better projection overall because it avoids too much distortion on the edges. However, that comes at the cost of some more artefacts in the central parts of the image. So when cropping into the image as I did above, the equirectangular may actually work best.
After quite some time fiddling around with FFmpeg and trying to understand the various parts of the new v360 function, I can conclude that the original equidistant projection is probably the best one to use for my analysis. The other projections probably work better for various types of 3D projections. Still, it was useful to learn how to run these processes using FFmpeg. This will surely come in handy when I am going to process a bunch of these files in the near future.