FFmpeg

I am recording a lot of short videos these days for my sound actions project. Sometimes the recordings end up being rotated, which is based on the orientation sensor (probably the gyroscope) of my mobile phone. This rotation is not part of the recorded video data, it is just information written into the header of the MPEG file. That also means that it is possible to change the rotation without recoding the file. It is possible to see the rotation by looking at the metadata of a file: ...

This week we started up our new online course, Motion Capture: The Art of Studying Human Activity, and we are also rerunning Music Moves: Why Does Music Make You Move? for the seventh time. Most of the material for these courses is premade, but we record a new wrap-up video at the end of each week. This makes it possible to answer questions that have been posed during the week and add some new and relevant material. ...

I often want to create motion videos, that is, videos that only show what changed between frames. Such videos are nice to look at, and so-called “frame differencing” is also the start point for many computer vision algorithms. We have made several tools for creating motion videos (and more) at the University of Oslo: the standalone VideoAnalysis app (Win/Mac) and the different versions of the Musical Gestures Toolbox. These are all great tools, but sometimes it would be nice also to create motion videos in the terminal using FFmpeg. ...

I have previously written about how it is possible to “flatten” a Ricoh Theta+ recording using FFmpeg. Now, I have spent some time exploring how to process some recordings from a Garmin VIRB camera. Some hours of recordings The starting point was a bunch of recordings from our recent MusicLab Copenhagen featuring the amazing Danish String Quartet. A team of RITMO researchers went to Copenhagen and captured the quartet in both rehearsal and performance. We have data and media from motion capture, eye tracking, physiological sensing, audio, video, and more. The plan is to make it all available on OSF. ...

I had a period in the early 2000s when I made a few pieces in the genre of electroacoustic music (“tape music”). One of these was a 4-channel piece called Peer Peer, made for an exhibition opening at Henie-Onstad Kunstsenter in 2003. The piece was based on a montage of short snippets sampled from Edvard Grieg’s music to the play Peer Gynt by Henrik Ibsen. Someone asked about the piece the other day, so I decided to find the files. ...

Typical video files, such as MP4 files with H.264 compression, are usually small in size and with high visual quality. Such files are suitable for visual inspection but do not work well for video analysis. In most cases, computer vision software prefers to work with raw data or other compression formats. The Musical Gestures Toolbox for Matlab works best with these file types: Video: use.jpg (Motion.jpg) as the compression format. This compresses each frame individually. Use .AVI as the container, since this is the one that works best on all platforms. Audio: use uncompressed audio (16-bit PCM), saved as .WAV files (.AIFF usually also works fine). If you need to use compression, MP3 compression (MPEG-1, Layer 3) is still more versatile than AAC (used in .MP4 files). If you use a bitrate of 192 Kbs or higher, you should not get too many artefacts. Many people ask me how to convert from typical MP4 files (with H.264 video compression and AAC audio compression). The easiest solution (I think) is to use FFMPEG, the versatile command-line utility. Here is a oneliner that will convert from an .MP4 file into a .AVI file with.jpg and PCM audio: ...

Here is another FFmpeg-related blog post, this time to explain how to rotate a video using the command-line tool FFmpeg. There are two ways of doing this, and I will explain both in the following. Rotation in metadata The best first try could be to make the rotation by only modifying the metadata in the file. This does not work for all file types, but should work for some (including .mp4) files. ...

I have previously written about how to trim video files with FFmpeg. It is also easy to crop a video file. Here is a short how-to guide for myself and others. Cropping is not the same as trimming This may be basic, but I often see the concepts of cropping and trimming used interchangeably. So, to clarify, trimming a video file means making it shorter by removing frames in the beginning and/or end. That is not the same as cropping a video file, which only selects a particular part of the video for export. ...

Sometimes, there is a need to convert an audio file into a blank video file with an audio track. This can be useful if you are on a system that does not have a dedicated audio player but a video player (yes, rare, but I work with odd technologies…). Here is a quick recipe FFmpeg to the rescue When it comes to converting from one media format to another, I always turn to FFmpeg. It requires “coding” in the terminal, but usually, it is only necessary to write a oneliner. When it comes to converting an audio file (say in .WAV format) to a blank video file (for example, a .AVI file), this is how I would do it: ...

We are organizing the Rhythm Production and Perception Workshop at RITMO next week. As mentioned in another blog post, we have asked presenters to send us pre-recorded videos. They are all available on the workshop page. During the workshop, we will play sets of videos in sequence. When doing a test run today, we discovered that the sound levels differed wildly between files. There is clearly the need for normalizing the sound levels to create a good listener experience. ...