MusicTestLab as a Testbed of Open Research

Many people talk about “opening” the research process these days. Due to initiatives like Plan S, much has happened when it comes to Open Access to research publications. There are also things happening when it comes to sharing data openly (or at least FAIR). Unfortunately, there is currently more talking about Open Research than doing. At RITMO, we are actively exploring different strategies for opening our research. The most extreme case is that of MusicLab. In this blog post, I will reflect on yesterday’s MusicTestLab – Slow TV.

About MusicLab

MusicLab is an innovation project by RITMO and the University Library. The aim is to explore new methods for conducting research, research communication and education. The project is organized around events: a concert in a public venue, which is also the object of study. The events also contain an edutainment element through panel discussions with world-leading researchers and artists, as well as “data jockeying” in the form of live data analysis of recorded data.

We have carried out 5 full MusicLab events so far and a couple of in-between cases. Now we are preparing for a huge event in Copenhagen with the Danish String Quartet. The concert has already been postponed once due to corona, but we hope to make it happen in May next year.

The wildest data collection ever

As part of the preparation for MusicLab Copenhagen, we decided to run a MusicTestLab to see if it is at all possible to carry out the type of data collection that we would like to do. Usually, we work in the fourMs Lab, a custom-built facility with state-of-the-art equipment. This is great for many things, but the goal of MusicLab is to do data collection in the “wild”, which would typically mean a concert venue.

For MusicTestLab, we decided to run the event on the stage in the foyer of the Science Library at UiO, which is a real-world venue that gives us plenty of challenges to work with. We decided to bring a full “package” of equipment, including:

  • infrared motion capture (Optitrack)
  • eye trackers (Pupil Labs)
  • physiological sensors (EMG from Delsys)
  • audio (binaural and ambisonics)
  • video (180° GoPros and 360° Garmin)

We are used to working with all of these systems separately in the lab, but it is more challenging when combining them in an out-of-lab setting, and with time pressure on setting everything up in a fairly short amount of time.

Musicians on stage with many different types of sensors on, with RITMO researchers running the data collection and a team from LINK filming.

Streaming live – Slow TV

In addition to actually doing the data collection in a public venue, where people passing by can see what is going on, we decided to also stream the entire setup online. This may seem strange, but we have found that many people are actually interested in what we are doing. Many people also ask about how we do things, and this was a good opportunity to show people the behind-the-scenes of a very complex data collection process. The recording of the stream is available online:

To make it a little more watcher-friendly, the stream features live commentary by myself and Solveig Sørbø from the library. We talk about what is going on and make interviews with the researchers and musicians. As can be seen from the stream, it was a quite hectic event, which was further complicated by corona restrictions. We were about an hour late for the first performance, but we managed to complete the whole recording session within the allocated time frame.

The performances

The point of the MusicLab events is to study live music, and this was also the focal point of the MusicTestLab, featuring the very nice, young student-led Borealis String Quartet. They performed two movements of Haydn’s Op. 76, no. 4 «Sunrise» quartet. The first performance can be seen here (with a close-up of the motion capture markers):

The first performance of Haydn’s string quartet Op. 76, no. 4 (movements I and II) by the Borealis String Quartet.

Then after the first performance, the musicians took off the sensors and glasses, had a short break, and then put everything back on again. The point of this was for the researchers to get more experience with putting everything on properly. From a data collection point of view, it is also interesting to see how reliable the data are between different recordings. The second performance can be seen here, now with a projection of the gaze from the violist’s eye-tracking glasses:

The second performance of Haydn’s string quartet Op. 76, no. 4 (movements I and II) by the Borealis String Quartet.

A successful learning experience

The most important conclusion of the day was that it is, indeed, possible to carry out such a large and complex data collection in an out-of-lab setting. It took an hour longer than expected to set everything up, but it also took an hour less to take everything down. This is valuable information for later. We also learned a lot about what types of clamps, brackets, cables, etc., that are needed for such events. Also useful is the experience of calibrating all the equipment in a new and uncontrolled environment. All in all, the experience will help us in making better data collections in the future.

Sharing with the world

Why is it interesting to share all of this with the world? RITMO is a Norwegian Centre of Excellence, which means that we get a substantial amount of funding for doing cutting-edge research. We are also in a unique position to have a very interdisciplinary team of researchers, with broad methodological expertise. With the trust we have received from UiO and our many funding agencies, we, therefore, feel an obligation to share as much as possible of our knowledge and expertise with the world. Of course, we present our findings at the major conferences and publish our final results in leading journals. But we also believe that sharing the way we work can help others.

Sharing our internal research process with the world is also a way of improving our own way of working. Having to explain what you do to others help to sharpen your own thinking. I believe that this will again lead to better research. We cannot run MusicTestLabs every day. Today all the researchers will copy all the files that we recorded yesterday and start on the laborious post-processing of all the material. Then we can start on the analysis, which may eventually lead to a publication in a year (or two or three) from now. If we do end up with a publication (or more) based on this material, everyone will be able to see how it was collected and be able to follow the data processing through all its chains. That is our approach to doing research that is verifiable by our peers. And, if it turns out that we messed something up, and that the data cannot be used for anything, we have still learned a lot through the process. In fact, we even have a recording of the whole data collection process so that we can go back and see what happened.

Other researchers need to come up with their approaches to opening their research. MusicLab is our testbed. As can be seen from the video, it is hectic. Most importantly, though, is that it is fun!

RITMO researchers transporting equipment to MusicTestLab in the beautiful October weather.

Moving to a new building

I have not been very good at blogging recently. This is not because nothing is happening, but rather because so much is happening that I don’t have time to write about it.

One of these things is the startup of RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, that I am co-directing with Anne Danielsen. We got the funding last year, and have spent the year in planning, preparing and now executing the startup. This includes moving to a different building – Harald Schjelderups Hus – on the northwest corner of the Blindern campus.

The fourMs lab is also moving, and right now everything is in boxes while we are waiting for the construction work to be done. Meanwhile, the new MCT master’s programme is moving into the old facilities of the fourMs lab, a very nice re-use of a great lab space.

What is almost in place, however, is my new office. After several days of packing, all my stuff was moved today, and I am looking forwards to getting everything set up. 

Come work with me! Lots of new positions at University of Oslo

I recently mentioned that I have been busy setting up the new MCT master’s programme. But I have been even more busy with preparing the startup of our new Centre of Excellence RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion. This is a large undertaking, and a collaboration between researchers from musicology, psychology and informatics. A visual “abstract” of the centre can be seen in the figure to the right.

Now we are recruiting lots of new people for the centre, so please apply or forward to people you think may be interested:

And we’re off: RITMO Centre for Interdisciplinary Studies in Rhythm, Time, and Motion

I am happy to announce that RITMO Centre for Interdisciplinary Studies in Rhythm, Time, and Motion officially started last week. This is a new centre of excellence funding by the Research Council of Norway.

Even though we have formally taken off, this mainly means that the management group has started to work. Establishing a centre with 50-60 researchers is not done in a few days, so we will more or less spend the coming year to get up to speed. The plan is that the faculty group will begin working together from January, while in parallel recruiting PhD and postdoctoral fellows. We aim at moving into our new spaces and having most of the people in place by August 2018, and that is also when we will have the kick-off party.

At least we now have a small web page up and running, and more content will be added as we move along. Here is a short summary of what we will be working on:

RITMO is an interdisciplinary research centre focused on rhythm as a structuring mechanism for the temporal dimensions of human life.
The research will be highly interdisciplinary, combining methods from musicology, psychology and informatics to study rhythm as a fundamental property that shapes and underpins human cognition, behaviour and culture.

Rhythm is omnipresent in human life, as we walk, talk, dance and play; as we tell stories about our past; and as we predict the future. Rhythm is also central to human biology, from the oscillations of our nervous system to our heartbeats, breathing patterns and longer chronobiological cycles. As such, it is a key aspect of human action and perception that is in complex interplay with the various cultural, biological and mechanical rhythms of the world.

RITMO will undertake research on rhythm in human action and perception, using music, motion and audio-visual media as empirical points of departure. Our core idea is that the human ability to experience the world and our actions as rhythmic, points to a basic cognitive mechanism that is in itself rhythmic in nature. The vision of RITMO is to understand more about this cognitive mechanism, and through this generate ground-breaking knowledge about the ways in which humans structure and understand the temporal dimensions of their life.

The centre is interdisciplinary and will combine perspectives and methods from music and media studies, philosophy and aesthetics, cognitive neuroscience, and informatics, using state-of-the-art technologies for motion capture, neuroimaging, pupillometry and robotics.

RITMO is to reveal the basic cognitive mechanism(s) underlying human rhythm, using music, motion and audiovisual media as empirical points of departure.

The research will be highly interdisciplinary, combining methods from musicology, psychology and informatics to study rhythm as a fundamental property that shapes and underpins human cognition, behaviour and culture.

New Centre of Excellence: RITMO

The new centre directors: Anne Danielsen (years 1–5) and Alexander Refsum Jensenius (years 6-10).

I am happy to announce that the Research Council of Norway has awarded funding to establish RITMO –  Centre of Excellence for Interdisciplinary Studies in Rhythm, Time and Motion. The centre is a collaboration between Departments of Musicology, Psychology and Informatics at University of Oslo.

Project summary

Rhythm is omnipresent in human life, as we walk, talk, dance and play; as we tell stories about our past; and as we predict the future. Rhythm is also central to human biology, from the microoscillations of our nervous system to our heartbeats, breathing patterns and longer chronobiological cycles (or biorhythms). As such, it is a key aspect of human action and perception that is in complex interplay with the various cultural, biological and mechanical rhythms of the world.

The vision behind RITMO is to reveal the basic cognitive mechanism(s) underlying human rhythm, using music, motion and audiovisual media as empirical points of departure. No other interdisciplinary research environment has focused solely on rhythm and its direct and indirect impacts before. Given the fundamental role of rhythm in human life, such an endeavour is long overdue.

RITMO will undertake research on rhythm in human action and perception, and on the aesthetic and cultural ‘texts’ that such processes elicit. This venture will benefit from the combined perspectives of the humanities, cognitive neuroscience, social sciences and informatics. Now is the right time to establish such a centre, because we can finally explore some of the larger questions of the humanities via state-of-the-art technologies for motion capture, neuroimaging, pupillometry and robotics. Such a research strategy is as novel as it is essential to any engagement with the impact of human rhythm. RITMO will generate groundbreaking knowledge about the structuring and understanding of the temporal dimensions of human life. As such, it will change how we view human cognition and supply a cornerstone for the future exploitation of rhythm in applications for well-being and rehabilitation.