New Book Chapter: Gestures in ensemble performance

I am happy to announce that Cagri Erdem and I have written a chapter titled “Gestures in ensemble performance” in the new book Together in Music: Coordination, Expression, Participation edited by Renee Timmers Freya Bailes, and Helena Daffern.

Video Teaser

For the book launch, Cagri and I recorded a short video teaser:

Abstract

The more formal abstract is:

The topic of gesture has received growing attention among music researchers over recent decades. Some of this research has been summarized in anthologies on “musical gestures”, such as those by Gritten and King (2006), Godøy and Leman (2010), and Gritten and King (2011). There have also been a couple of articles reviewing how the term gesture has been used in various music-related disciplines (and beyond), including those by Cadoz and Wanderley (2000) and Jensenius et al. (2010). Much empirical work has been performed since these reviews were written, aided by better motion capture technologies, new machine learning techniques, and a heightened awareness of the topic. Still there are a number of open questions as to the role of gestures in music performance in general, and in ensemble performance in particular. This chapter aims to clarify some of the basic terminology of music-related body motion, and draw up some perspectives of how one can think about gestures in ensemble performance. This is, obviously, only one way of looking at the very multifaceted concept of gesture, but it may lead to further interest in this exciting and complex research domain.

Ten years after Musical Gestures

We began writing this ensemble gesture chapter in 2020, about ten years after the publication of the chapter Musical gestures: Concepts and methods in research. That musical gesture chapter has, to my surprise, become my most-cited publication to date. When I began working on the topic of musical gestures with Rolf Inge Godøy back in the early 2000s, it was still a relatively new topic. Most music researchers I spoke to didn’t understand why we were interested in the body.

Fast forward to today, and it is hard to find music researchers that are not interested in the body in one way or another. So I am thrilled about the possibility of expanding some of the “old” thoughts about musical gestures into the ensemble context in the newly published book chapter.

New publication: “To Gesture or Not” (NIME 2014)

This week I am participating at the NIME conference, organised at Goldsmiths, University of London. I am doing some administrative work as chair of the NIME steering committee, and I am also happy to present a paper tomorrow:

Title
To Gesture or Not? An Analysis of Terminology in NIME Proceedings 2001–2013

Links
Paper (PDF)
Presentation (HTML)
Spreadsheet with summary of data (ODS)
OSX shell script used for analysis

Abstract
The term ‘gesture’ has represented a buzzword in the NIME community since the beginning of its conference series. But how often is it actually used, what is it used to describe, and how does its usage here differ from its usage in other fields of study? This paper presents a linguistic analysis of the motion-related terminology used in all of the papers published in the NIME conference proceedings to date (2001– 2013). The results show that ‘gesture’ is in fact used in 62 % of all NIME papers, which is a significantly higher percentage than in other music conferences (ICMC and SMC), and much more frequently than it is used in the HCI and biomechanics communities. The results from a collocation analysis support the claim that ‘gesture’ is used broadly in the NIME community, and indicate that it ranges from the description of concrete human motion and system control to quite metaphorical applications.

Reference
Jensenius, A. R. (2014). To gesture or not? An analysis of terminology in NIME proceedings 2001–2013. In Proceedings of the International Conference on New Interfaces For Musical Expression, pages 217–220, London.

BibTeX

@inproceedings{Jensenius:2014c,
    Address = {London},
    Author = {Jensenius, Alexander Refsum},
    Booktitle = {Proceedings of the International Conference on New Interfaces For Musical Expression},
    Pages = {217--220},
    Title = {To Gesture or Not? {A}n Analysis of Terminology in {NIME} Proceedings 2001--2013},
    Year = {2014}}

Definitions: Motion, Action, Gesture

I have been discussing definitions of the terms motion/movement, action and gesture several times before on this blog (for example here and here). Here is a summary of my current take on these three concepts:

Motion: displacement of an object in space over time. This object could be a hand, a foot, a mobile phone, a rod, whatever. Motion is an objective entity, and can be recorded with a motion capture system. A motion capture system could be anything from a simple slider (1-dimensional), to a mouse (2-dimensional), to a camera-based tracking system ((3-dimensional)  or an inertial system (6-dimensional: 3D position and 3D orientation). I have previously also discussed the difference between motion and movement. Since motion is a continuous phenomenon, it does not make sense to talk about it in plural form: “motions”. Then it makes more sense to talk about one or more motion sequences, but most probably it makes even more sense to talk about individual actions.

Action: a goal-directed motion (or force) sequence, for example picking up a stone from the ground, playing a piano tone. Actions may have a clear beginning and end, but they may also overlap due to coarticulation, such as when playing a series of tones on the piano. This uncertainty as to how actions should be segmented (or chunked), is what makes them subjective entities. As such, I do not think it is possible to measure an action directly, since there is no objective measure for when an action begins or ends, or how it is organised in relation to other actions. But, based on knowledge about human cognition, it is possible to create systems that can estimate various action features based on measurements of motion.

Gesture: the meaning being expressed through an action or motion. A gesture is not the same as action or motion, although it is related to both of them. As such, a gesture can be seen as a semiotic sign, in which the meaning is conveyed through an action, but it is highly subjective and dependent on the cultural context in which the action is carried out. Also, the same meaning can be conveyed through different types of physical actions. For example, the meaning you convey when you wave “good-bye” to someone may be independent of whether you do it with the left or the right arm, the size of the action, etc.

Unfortunately, with the popularity of motion and gesture studies over the last years, I see that many people use the term gesture more or less synonymously to action or motion. This is particularly the case in the field of “gesture recognition” in various versions of human-computer interaction (HCI).  I think it is unfortunate because we loose the precision with which we can describe the three different phenomena. If we track continuous motion in time and space, it is “motion tracking”. If we aim at recognising certain physical patterns in time and space, I would call it “action recognition” unless we are looking for some meanings attached to the actions. “Gesture recognition” I would only use if we actually recognise the meaning attached to some actions or motion. An example here would be to recognise the emotional quality of the performance  of a violinist. That, however, is something very different than tracking the bowing style.

movement-action
An illustration of my definition of the difference between motion and action