Documentation of the NIME project at Norwegian Academy of Music

From 2007 to 2011 I had a part-time research position at the Norwegian Academy of Music in a project called New Instruments for Musical Exploration, and with the acronym NIME. This project was also the reason why I ended up organising the NIME conference in Oslo in 2011.

The NIME project focused on creating an environment for musical innovation at the Norwegian Academy of Music, through exploring the design of new physical and electronic instruments. We were three people involved in the project, percussionist/electro-improviser Kjell Tore Innervik, composer Ivar Frounberg, and myself, and we had a great time together in creating and performing with a number of different new instruments.

A slogan for the project was to create instrument “for the many and for the few”. The “for the many” part we approached through the creation of Oslo Laptop Orchestra and Oslo Mobile Orchestra, and the creation of a series of music balls. The “for the few” part was more specifically targeted at creating specific instruments for professional musicians. Some of these were glass instruments, and here we also did some historic and analytic studies that were presented at NIME 2010.

As an artistic research project we were also careful about documenting all the processes we were involved in, and we also ended up creating a final series of video documentaries reflecting on the process and the artistic outcomes. Kjell Tore has written more about all of this on his own web page. Here I would like to mention three short documentaries we created, reflecting on the roles of technologist, performer, and composer in the project. Creating these documentaries was in itself an interesting exercise. As an academic researcher, I am used to writing formal research papers about my findings. However, as artistic researchers in the NIME project, we all felt that a more discussion-based reflection was more suitable. The documentaries are, unfortunately, only in Norwegian, but we hope to be able to include subtitles in English at some point.

Visualisations of a timelapse video

Yesterday, I posted a blog entry on my TimeLapser application, and how it was used to document the working process of the making of the sculpture Hommage til kaffeselskapene by my mother. The final timelapse video looks like this:

Now I have run this timelapse video through my VideoAnalysis application, to see what types of analysis material can come out of such a video.

The average image displays a “summary” of the entire video recording, somehow similar to an “open shutter” in traditional photography. This image allows for seeing what has been moving and what has not been moving throughout the entire sequence.

Average image
Average image 

The motion average image is somehow similar to the average image, but it summarises the motion images through the entire sequence, that is, only the parts of the image that changed.

Motion average image
Motion average image

What I call a motion history image, is the motion average image overlaid only a single frame from the original video. I typically create such motion history images using both the first and last frames of the video, as can be seen below.

Motion history image, based on first video frame
Motion history image, based on first video frame
Motion history image, based on last video frame
Motion history image, based on last video frame

Finally, I have also created both horisontal and vertical motiongrams of the timelapse video. The horisontal motiongram displays the vertical motion, which in this case is how the sculptor moved back and forth when sitting at the table. The edge of the table can be seen as the “stripe” running throughout the image.

Horisontal motiongram, displaying vertical motion
Horisontal motiongram, displaying vertical motion

The vertical motiongram, on the other hand, displays horisontal motion, that is, how the artist moved sideways throughout the process. Here it is very interesting to note the rhythmic swaying pattern, as the sculptor moved back and forth in what seems to be a periodic pattern.

Vertical motiongram, displaying horisontal motion
Vertical motiongram, displaying horisontal motion

I also have some more motion data, which it will be interesting to study in more detail in Matlab.

Sverm video #3

Video artist Lavasir Nordrum hast just posted the third of four short movies created together with the Sverm group. The first short movie was titled Micromovements, and the second was titled Microsounds. This month’s short movie is called Excitation, and is focused on the first half of an even or action. This will be followed by a short movie called Resonance to be released on 1 January.

Sverm video #1

For the last couple of years I have been involved in an artistic research project called Sverm, in which we investigate the artistic potential of bodily micromovements and microsound. We are currently working towards a series of intimate lab performances in the end of November.

As a side-project to the performances, we are also working with video artist Lavasir Nordrum, on the making of four short videos documenting the four main parts of the project: micromovement, microsound, excitation, resonance.

The first of the videos are now ready, focusing on the topic of micromovement, and featuring Kari Anne Vadstensvik Bjerkestrand, Victoria Johnson and myself.

Trond Lossius’ fellowship report

I spent my flight to Montreal (which became much longer than I expected when I was rescheduled through Chicago) reading Trond Lossius’ report for the Fellowship in the arts program. He addresses a number of interesting topics:

Commenting on the necessity for carrying out research for instead of on art, he discusses the concept of “art as code”:

It is not only a question of developing tools. [..] Programming code becomes a meta-medium, and creating the program is creating the art work.

This idea resonates throughout the text, and is followed by a discussion of the importance of craftsmanship in electronic arts:

This text might leave the impression that I am very concerned about the technical aspects of the works I create, with the potential risk of the works to end up having a certain geek factor. I do spend a lot of time and energy researching technical solutions that might help me achieve what I want, but they are primarily means to an end. I consider my practice to somewhat resemble how a professional pianist works. Several hours a day are spent rehearsing scales and other exercises in order to develop and maintain a high level of technical skills. But the development of technical skills is not a purpose in itself, and it would be out of question to play scales and etudes in concerts. The skills are required in order to be on top of the material one is working on, and to be able to articulate oneself artistically without technical limitations becoming a hindrance reducing the impact that message can be delivered with.

I very much agree with this. The tools we use certainly colours what we produce, whether it be scientific or artistic work. Thus, developing tools can often be considered to be the most important outcome of a research process. However, as Trond discuss in the end of the dissertation, such development might often be problematic in collaborative projects:

In stage productions it is very hard to find this kind of time and space alone for development, testing, fool-proofing and debugging. […] Coming up with a new idea might take three seconds, but developing the patch required to realize it might take three hours or even three days, and in the mean-time everything risk coming to a crawl. Most of the time it is hard to say how long the development or alteration of a process or algorithm will take. […] This makes it hard for others to plan other tasks to work on in the meantime, and the actors, musicians or dancers to often end up waiting. When that happens for the second time in a day, energy and motivation is drained, and they struggle to mobilize the presence required for their contributions to the project. […] It is a continuous struggle to find the right balance between working fast so that the general flow of the production is not hindered more than necessary, and investing time in quality assurance, making sure that patches are sufficiently flexible, structured, documented and tested so that they do not cause problems further down the road. I have seen repeatedly when working with others that the importance of this has to be learned the hard way.

This is another point where I totally agree with Trond. Obviously, by developing better and more flexible tools, it should be possible to reduce the time necessary for patching and development during rehearsals. But this is really part of what a creative technologist will have to deal with, and it is important for others to realise that this is actually part of the creative process, in much the same way as traditional musicians need to warm up and practice their scales.