Computational

Over the past few weeks, I have read several newspaper articles and seen numerous social media postings from researchers expressing skepticism about using AI in research. It is fine that some people prefer not to use new tools, but I don’t see why that should hinder the rest of us from exploring the possibilities that are out there. Many researchers do use AI tools, which I think is perfectly fine. The problem is that not many people talk about how they use the tools and reflect more on how they work. I think it is imperative that professors explore AI in various ways to see what works and what does not. After all, we are going to teach upcoming generations and lead the way in both development and use. We then need hands-on experience and qualified opinions. ...

Yesterday, I wrote about how my kinesphere looked different from the ideals of Vitruvius, iconized through the drawing Vitruvian Man by Leonardo da Vinci. In particular, my legs and arms do not fit within a circle like in Leonardo’s painting. My kinesphere looks different from Leonardo da Vinci’s Vitruvian Man. The Wikipedia page lists a quite extensive description of the ideal measurements of a man: ...

As part of my year-long StillStanding project, I post an average image of the spherical video recordings on Mastodon daily. Average images The average image is similar to an “open shutter” technique in photography; it overlays all the frames in a video. The result is an image that shows the most prominent parts of the video recording. This is ideal for my StillStanding recordings, because the technique effectively “removes” objects that appear in the recording for a short period of time. That means that people, cars, and animals do not appear. Since I stand still, the average image may look like a photography, but you can see that my face is blurred. ...

There is always a need to add fade-in and fade-out to audio tracks. Here is a way of doing it for a bunch of video files. It may come in handy with the audio normalization script I have shown previously. That script is based on continuously normalizing the audio, which may result in some noise in the beginning and end (because there is little/no sound in those parts, hence they are normalized more). ...

Today, I recorded Sound Action 194 - Rolling Dice as part of my year-long sound action project. The idea has been to do as little processing as possible to the recordings. That is because I want to capture sounds and actions as naturally as possible. The recorded files will also serve as source material for both scientific and artistic explorations later. For that reason, I only trim the recordings non-destructively using FFmpeg. ...

I am recording a lot of short videos these days for my sound actions project. Sometimes the recordings end up being rotated, which is based on the orientation sensor (probably the gyroscope) of my mobile phone. This rotation is not part of the recorded video data, it is just information written into the header of the MPEG file. That also means that it is possible to change the rotation without recoding the file. It is possible to see the rotation by looking at the metadata of a file: ...

I often want to create motion videos, that is, videos that only show what changed between frames. Such videos are nice to look at, and so-called “frame differencing” is also the start point for many computer vision algorithms. We have made several tools for creating motion videos (and more) at the University of Oslo: the standalone VideoAnalysis app (Win/Mac) and the different versions of the Musical Gestures Toolbox. These are all great tools, but sometimes it would be nice also to create motion videos in the terminal using FFmpeg. ...

I have previously written about how it is possible to “flatten” a Ricoh Theta+ recording using FFmpeg. Now, I have spent some time exploring how to process some recordings from a Garmin VIRB camera. Some hours of recordings The starting point was a bunch of recordings from our recent MusicLab Copenhagen featuring the amazing Danish String Quartet. A team of RITMO researchers went to Copenhagen and captured the quartet in both rehearsal and performance. We have data and media from motion capture, eye tracking, physiological sensing, audio, video, and more. The plan is to make it all available on OSF. ...

At the first “Tour de RITMO”, a retreat for the whole centre, we had a group work session that resulted in the development of our code of conduct. Now, with a new group of doctoral and postdoctoral fellows that have just started on their RITMO journey, it was time to refresh the “rules”. The new version is not dramatically different than the old one, but the discussions behind it were lively. The most important change is the addition of a sixth element on ethics. ...

Typical video files, such as MP4 files with H.264 compression, are usually small in size and with high visual quality. Such files are suitable for visual inspection but do not work well for video analysis. In most cases, computer vision software prefers to work with raw data or other compression formats. The Musical Gestures Toolbox for Matlab works best with these file types: Video: use.jpg (Motion.jpg) as the compression format. This compresses each frame individually. Use .AVI as the container, since this is the one that works best on all platforms. Audio: use uncompressed audio (16-bit PCM), saved as .WAV files (.AIFF usually also works fine). If you need to use compression, MP3 compression (MPEG-1, Layer 3) is still more versatile than AAC (used in .MP4 files). If you use a bitrate of 192 Kbs or higher, you should not get too many artefacts. Many people ask me how to convert from typical MP4 files (with H.264 video compression and AAC audio compression). The easiest solution (I think) is to use FFMPEG, the versatile command-line utility. Here is a oneliner that will convert from an .MP4 file into a .AVI file with.jpg and PCM audio: ...