Yesterday, PhD fellow Mojtaba Karbassi presented his research on impedance control in robotic drumming at RITMO. I will surely get back to discussing more of his research later. Today, I wanted to share the analysis of one of the videos he showed. Mojtaba is working on developing a robot that can play a double stroke drum roll. To explain what this is, he showed this video he had found online, made by John Wooton:
The double stroke roll is a standard technique for drummers, but not everyone manages to perform it as evenly as in this example. I was eager to have a look at the actions in a little more detail. We are currently beta-testing the next release of the Musical Gestures Toolbox for Python, so I thought this video would be a nice test case.
I started the analysis by extracting the part of the video where he is showing the complete drum roll. Next, I generated a motion video of this segment:
This is already fascinating to look at. Since the background is removed, only the motion is visible. Obviously, the framerate of the video is not able to capture the speed that he plays with. I was therefore curious about the level of detail I could achieve in the further analysis.
Before delving into the visualization of the video file, I made a spectrogram of the sound:
If you are used to looking at spectrograms, you can quite clearly see the change in frequency as the drummer is speeding up and then slowing down again. However, a tempogram of the audio is even clearer:
Here you can really see the change in both the frequency and the onset strength. The audio is sampled at a much higher frequency (44.1 kHz) than the video (25 fps). Is it possible to see some of the same effects in the motion?
I then moved on to create a motiongram of the video:
There are two problems with this motiongram. First, the recording is composed of alternating shots from two different camera angles. These changes between shots can clearly be seen in the motiongram (marked with Camera 1 and 2). Second, this horizontal motiongram only reveals the vertical motion in the video image. Since we are here averaging over each row in the image, the motiongram shows both the left and right-hand motion. For such a recording, it is, therefore, more relevant to look at the vertical motiongram, which shows the horizontal motion:
In this motiongram, we can more clearly see the patterns of each hand. Still, we have the problem of the alternating shots. If we “zoom” in on the part called Camera 2b, it is possible to see the evenness of the motion in the most rapid part:
I also find it fascinating to “zoom” in on the part called Camera 2c, which shows the gradual slow-down of motion:
Finally, let us consider the slowest part of the drum roll (Camera 1d):
Here it is possible to see the beauty of the double strokes very clearly.
I have never thought about how so-called plug-in-power microphones actually work. Over the years, I have used several of them for various applications, including small lavalier microphones for cameras and mobile phones. The nice thing about plug-and-play devices is that they are, well, plug and play. The challenge, however, is when they don’t work. Then it is time to figure out what is actually going on. This is the story of how I managed to use a Røde SmartLav+ lavalier microphone with a Zoom Q8 recorder.
When speaking about large (normal) microphones, we typically differentiate between two types: dynamic and condenser. The dynamic microphones are typically used for singing and talking and don’t require any power. You can plug them into a mixer or sound card, and they will work. Dynamic microphones are very versatile and are great in that they often don’t lead to much feedback. The downside to this is that they don’t pick up soft sounds very well, so you need to speak/sing quite loudly directly into them to get a good signal.
Condenser microphones are much more sensitive and allow for picking up more details than dynamic ones. However, to make them work, condenser microphones need to be supplied with 48-volt power, often called phantom power. Most mixers and sound cards have the ability to serve phantom power over an XLR connection, so it is usually no problem to get a good signal from a condenser microphone. Since there is only one connection type (XLR) and one type of power (48 volts) things are fairly straight forward (yes, I know, there are some exceptions here, but it holds for most cases).
As I have been doing more video recording and streaming over the last years, I have also gotten used to working with lavalier microphones. These are the tiny microphones you can place on your shirt to get a good sound quality when speaking on video. Over the years, I have been working with various microphones that come bundled as part of wireless packages. You have a transmitter to which you attach the microphone and a receiver to plug into your video camera. The transmitter and receiver run on batteries, but I have always thought that the power was used for the wireless transmission. Now I have learned that these microphones receive power from the transmitter. That is quite obvious when you think about it. After all, they pick up sound like a large condenser microphone. But I never really thought about them as powered microphones before.
I nowadays often use my mobile phone for quick and dirty video recordings. This works well for many things, and, as they say, the best camera is the one you bring with you. The sound, however, is less than optimal. I, therefore, wanted to use a lavalier microphone with my phone. Then the problems started.
It turns out that the world of lavalier microphones is much more complex than I would have imagined. To start with, there are numerous connectors for such microphones, including minijack plugs of different sizes (2.5 and 3.5 mm) and a number of rings (TS, TRS and TRRS), mini-XLR plugs with a different number of pins (TA-, TA-4, TA-5), in addition to Hirose, LEMO, and so on.
As I looked around the collection of my own lavalier microphones and also the ones we have in the lab, none of them had a 3.5 mm minijack connector that I could plug straight into my phone (yes, I still have a minijack plug on my phone!). So I quickly gave up and looked around on the web. Many people recommended the Røde SmartLav+, so I decided to get one to try out.
I liked the SmartLav+ so much (a comparison with some other devices) that I bought another one, some extender cables, and a small adapter to connect two of them to my phone simultaneously. Voila, and I have a nice and small kit for recording two people at a time. I have been using this to record many educational videos this last year, and it has worked very well. So if you want a small, simple, and (comparably) cheap setup to improve audio on your mobile phone recordings, you should get something like this. I should say that I have no particular reason for recommending the Røde SmartLav+ over other ones. Now I see that many people also recommend Shure MVL, which is probably equally good.
Connecting the SmartLav+ to a GoPro camera
I had been using the SmartLav+ with my phone for a while when I decided to try it with a GoPro 8. With the MediaMod accessory, it is possible to connect a microphone with a minijack plug. But plugging in the SmartLav+ does not work. This was when I started thinking more about the fact that the SmartLav+ has a so-called TRRS plug (as opposed to TRS and TS plugs).
In many consumer products, these three types are used for mono signals, stereo signals, and headsets (mono microphone + stereo output), respectively (although things are not always that easy).
To work with regular audio input (on the GoPro) the SmartLav+ signal needs to be converted from TRRS to TRS. Fortunately, there are adaptors for this, and it turned out that I had a few lying around in my office. I still decided to buy a Røde SC3 because it has a grey colour on the TRRS side, making it easier to see the connector type.
When I plugged in the microphone (with adaptor) to the GoPro, it worked nicely right out of the box. I, therefore, didn’t think much about the need to power the microphone. I have later learned from DC Rainmaker that the GoPro actually has a setting for choosing between different types of microphone inputs:
The list above says that the GoPro defaults to non-powered mics, but my camera defaults to plug-in-power. They might have changed things along the way.
Connecting the SmartLav+ to a Zoom Q8 recorder
When I tried to connect the SmartLav+ to a Zoom Q8 recorder, I started having problems. First, I connected with a minijack-to-jack adaptor (with the TRRS-TRS adaptor in between). This resulted in no sound input on the Q8. I then switched to an XLR adaptor, but still no sound. I then took out a dynamic microphone to check that the Q8 input actually worked.
This was when I realized that SmartLav+ is actually a powered microphone. After reading up more on this and other lavalier microphones, I understand that I have had a big gap in my microphone knowledge. This is slightly embarrassing. After all, a professor of music technology should know about such things. To my excuse, perhaps, I would argue that lavalier microphones are not something that music technologists typically deal with. Most of the time, we work with large microphones and XLR cables. Such small microphones are typically used more for video recording and media production.
Embarrassments aside, I am primarily interested in finding a solution to my problem. How do I connect the SmartLav+, or any other powered minijack-microphone, with a sound recorder?
It turned out that Røde actually has a solution to the problem in the form of the minijack-to-XLR adapter VXLR+. This is not just a passive device converting from one to the other (I already had some of those lying around). No, this one actually converts the 48-volt power coming from the XLR cable to the 2.7 volts required by the SmartLav+. To complicate things, though, the adapter takes a TRS minijack as input, so it is also necessary to add the TRRS-TRS adapter in between. So after hooking it all up, and turning on phantom power, I now finally have a loud and clear sound on the Q8. The sound is not as good as with microphones like the DPA 4060, of course, but not bad for voice recordings.
One of the reasons I wanted to connect the SmartLav+ to the Zoom recorder in the first place, was to have a simple and portable setup for recording conversations with multiple people (4-8). Of course, I could set up an omnidirectional microphone or a stereo pair, but that wouldn’t give the type of intimate sound that I am looking for. In the lab, I could always set up many large microphones on stands, but that is not a very portable solution. So I was thinking about possibly connecting multiple lavalier microphones to a multichannel sound recorder instead. Now, I have found that this could actually work well. For example, a Zoom H8 with many lavalier microphones could be a nice and portable setup. While searching for such a setup, however, a different solution came to my attention.
Given that more and more people are using lavalier microphones these days, I was curious about the market for minijack-based mixers. Strangely enough, there aren’t many around and none from the big manufacturers. But one mixer seemed to pop up in various webshops: Just MIC IV by Maker Hart. It features four minijack inputs, and, most importantly, it can provide power to the microphones. In fact, it can provide both 48v and 1.5v.
This mixer looked like the perfect solution for my needs, so I decided to give it a try. After playing with it for a little while, I have found it to be almost exactly what I need. The functionality is great. It supplies power to the microphones. They should ideally get 2.7v, but the 1.5v supplied from the mixer seems to work fine. The panning is a rudimentary left-middle-right switch, which is not ideal but can place people in a stereo image. It only has a 2-channel output, so no multi-channel recording here. But it will suffice for quick recordings of four people.
The biggest problem with the Just MIC IV is that it picks up electric disturbances very easily. I often get an annoying buzz when connecting it to a wall socket. So I have ended up running it from a USB battery pack instead. Not ideal, but better than nothing.
After a lot of searching and testing I now know a lot more about lavalier microphones, different minijack congifurations, and interfacing possibilities. I still do not have an optimal solution for my needs, but am getting closer. Given that so many people are getting into sound recording these days, from podcasts to teaching, I think there is a potential market here for easy to use solutions. Products like the SmartLav+ has made it much easier to make good audio recordings on a mobile phone. I wish there was a decent small and simple mixer for such microphones. The Just MIC IV is almost there, but is too noisy. Any company out there that can make a small, solid, high-quality 8-channel mini-mixer?
The Musical Gestures Toolbox for Matlab (MGT) aims at assisting music researchers with importing, preprocessing, analyzing, and visualizing video, audio, and motion capture data in a coherent manner within Matlab.
Most of the concepts in the toolbox are based on the Musical Gestures Toolbox that I first developed for Max more than a decade ago. A lot of the Matlab coding for the new version was done in the master’s thesis by Bo Zhou.
This network brings together a group of internationally leading sound and music computing researchers from institutions in five Nordic countries: Aalborg University, Aalto University, KTH Royal Institute of Technology, University of Iceland, and University of Oslo. The network covers the field of sound and music from the “soft” to the “hard,” including the arts and humanities, and the social and natural sciences, as well as engineering, and involves a high level of technological competency.
At the University of Oslo we have one open PhD fellowship connected to the network, with application deadline 4 April 2018. We invite PhD proposals that focus on sound/music interaction with periodic/rhythmic human body motion (walking, running, training, etc.). The appointed candidate is expected to carry out observation studies of human body motion in real-life settings, using different types of mobile motion capture systems (full-body suit and individual trackers). Results from the analysis of these observation studies should form the basis for the development of prototype systems for using such periodic/rhythmic motion in musical interaction.
The appointed candidate will benefit from the combined expertise within the NordicSMC network, and is expected to carry out one or more short-term scientific missions to the other partners. At UiO, the candidate will be affiliated with RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion. This interdisciplinary centre focuses on rhythm as a structuring mechanism for the temporal dimensions of human life. RITMO researchers span the fields of musicology, psychology and informatics, and have access to state-of-the-art facilities in sound/video recording, motion capture, eye tracking, physiological measurements, various types of brain imaging (EEG, fMRI), and rapid prototyping and robotics laboratories.
I am involved in a student project which uses some Arduino Mega 2560 sensor interfaces in an interactive device. It has been a while since I worked with Arduinos myself, as I am mainly working with Belas these days. Also, I have never worked with the Mega before, so I had to look around a little to figure out how to set it up with Cycling ’74’s Max.
I have previously used Maxuino for interfacing Arduinos with Max. This is a general purpose tool, with a step by step approach to connecting to the Arduino and retrieving data. This is great when it works, but due to its many options, and a somewhat convoluted patching style, I found the patch quite difficult to debug when things did not work out of the box.
I then came across the opposite to Maxuino, a minimal patch showing how to get the data right off the serial port. As can be seen from the screenshot below, it is, in fact, very simple, although not entirely intuitive if you are not into this type of thing.
One thing is the connection, another is to parse the incoming data in a meaningful way. So I decided to fork a patch made by joesanford, which had solved some of these problems in a more easy to understand patching style. For this patch to work, it requires a particular Arduino sketch (both the Max patch and Arduino sketch are available in my forked version on github). I also added a small sound engine, so that it is possible to control an additive synthesis with the sensors. The steps to make this work is explained below.
The mapping from sensor data starts by normalizing the data from the 15 analog sensors to a 0.-1. range (by dividing by 255). Since I want to control the amplitudes of each of the partials in the additive synthesis, it makes sense to slightly reduce all of the amplitudes by multiplying each element with a decreasing figure, as shown here:
Then the amplitudes are interleaved with the frequency values and sent to an ioscbank~ object to do the additive synthesis.
Not a very advanced mapping, but it works for testing the sensors and the concept.