I am excited to announce the publication of a new paper that explores several themes I have been interested in recently: inverse and indirect mappings, embodied AI, and indoor environments. The paper, titled Inverse and indirect mappings in embodied AI systems in everyday environments, is written by PhD fellow Maham Riaz together with my former PhD fellow Cagri Erdem and myself.

Embodied AI

The paper is written for a special issue on “embodied AI”, a topic I have been interested in for many years. AI research has long focused largely on the “brain” of machines. While that has been successful, we will not be able to progress unless we also consider that machines need to have a body that can sense and act.

Just like we have seen an “embodied turn” in music research and most other disciplines, there is now a need to shift attention to embodiment in AI research. That is why we are working with robotic systems at RITMO, including the self-playing guitars, Dr Squiggles and Zrob. These don’t look like “normal” robots, but they are machine systems with sensing and acting bodies.

Everyday environments

In the AMBIENT project, we are exploring how everyday spaces function and how we can improve them. This is typically the domain of interior architects, but we approach the topic from a creative and musical perspective. How can we apply our knowledge of musical instrument development to create engaging interactive experiences in everyday life?

Our approach can be thought of as designing musicking technologies, interactive systems that are not necessarily designed for direct musical engagement yet still possess musical properties.

Inverse Interaction

Most interactive systems are based on direct control: you press a button, and something happens. It is a clear cause-and-effect relationship that gives us a sense of mastery. Inverse interaction, on the other hand, is the opposite: a design concept where reducing your input leads to an increased output from the system.

The self-playing guitars don’t respond to being strummed. Instead, they begin to create soundscapes when a person stands still before them. The less you move, the more the guitars generate a sound designed to “mimic human breathing.” This approach is impactful precisely because it subverts our expectations of control. It shifts the dynamic from command to exploration, inviting a dialogue where stillness is as important as action. For designers, this opens a new playbook for creating experiences of contemplation and discovery, rather than just task completion.

Passive musicking

In the paper, we introduce the concept of passive musicking: sound generation without intentional interaction. It builds on Christopher Small’s argument that music is not a “thing” but something you do.

As I wrote about in Sound Actions, musicking technologies build on this idea. They are not designed to play music for us but to let us take part in musicking through everyday actions. When the birdbox is triggered by someone walking down a hallway, a mundane action becomes a small sonic event; the person is musicking without intending to.

The idea we have been working on in AMBIENT is that simply being present in a room makes you a participant in the environment’s evolving soundscape. In our experiments, the Muzziball shifts its light patterns even when untouched and responds to the subtlest changes in orientation and acceleration when picked up. The self-playing guitars react to ambient sound and unintentional movement. Who is the performer here, us or the object? I would argue that it is both, and that this blurring of performer and perceiver is exactly what makes such systems interesting to study.

A fine line

Researching these topics makes it necessary to consider the ethics of our work. The same techniques that make a room subtly responsive can also be used for manipulative background interactions. There is already a history of that in Muzak and other commercial use of sounds. That is a line we will continue to discuss as we continue this research in the AMBIENT project and in MishMash.

Thanks to Grammarly for improving the language in this post.