Can AI replace humans?

Or, more specifically: can AI replace an artist? That is the question posed in a short documentary that I have contributed to for this year’s Research Days.

We were contacted before summer about trying to create a new song based on the catalogue of the Norwegian artist Ary. The idea was to use machine learning to generate the song. This has turned out to be an exciting project.

I was busy finishing the manuscript for my new book, so I wasn’t much involved in the development part myself. Most of the work has been done by a recent music technology graduate from the University of Oslo, Lars Monstad, gently supervised by my colleague Stefano Fasciani.

The project started with Ary sending us a bunch of her lyrics in text format and dumps from a digital audio workstation. This material was not really machine-readable/listenable, so Lars had to spend a great deal of time manually structuring and annotating it into a symbolic data set that could be used for training. The machine-learning part of the project involved generating lyrics following this approach, using an AI model that can already speak English. The melody was generated using Bachprop, based on a deep recurrent neural network. Then Lars put it all together into a final soundtrack that we played for Ary.

When talking about AI, I always find it important to highlight that humans are important for the final result. Yes, the machine makes something, but not without a lot of human guidance. The song played in the video (and the many other songs we also generated) was “composed” by a computer. However, Lars made many important decisions throughout the project: the initial preparation of the training material, the models used, the methods used, all the settings used, the selection of which lyrics and melodies to choose, and the final putting together of everything.

Last year, I was involved in several discussions about the potential challenges of using AI in music-making. I also wrote a blog post about some of the possibilities. While in theory AI can be used autonomously I believe that the most interesting is the meeting point between humans and machines. Providing artists like Ary with AI-based technologies can lead to exciting new music!

More research should be solid instead of novel

Novelty is often highlighted as the most important criterion for getting research funding. That a manuscript is novel is also a major concern for many conference/journal reviewers. While novelty may be good in some contexts, I find it more important that research is solid.

I started thinking about novelty versus solidity when I read through the (excellent) blog posts about the ISMIR 2021 Reviewing Experience. These blog posts deal with many topics, but the question about novelty caught my attention. Even though the numbers are small, it turned out that the majority of the survey respondents listed novelty as the most important selection criterion for the conference. This is not unique to ISMIR; I think many journals and conferences ask about novelty.

Defining novelty

Given that novelty is a criterion “everyone” considers all the time, few people discuss what it actually means. What does it actually mean that something is novel? Merriam-Webster suggests that it is “something new or unusual.” But what should be new or unusual? The questions? The answers? The methods?

Research is about contributing new knowledge to humankind. After all, it is not really any point in reinventing the wheel. Still, most research is incremental. We all stand on the shoulders of giants. New research questions spring out of the “future work” sections of our colleagues’ articles. Our methods are based on the refinement of disciplinary developments. Even so-called “groundbreaking” projects are incremental in nature if you scrutinize the details. Still, we have an idea that “something unheard of before” is ideal.

Research needs to be solid

My research is creative in both form and content. As such, many people think that my projects are novel in the sense of being new. I also work both multi- and interdisciplinary, which means that I don’t really fit well anywhere. That could also be considered novel in the sense of being unusual. Still, what I am doing is not particularly new or unusual. From my perspective, I am working incrementally—everything I am doing builds on other people’s work. True, I combine theories and methods from different fields. This makes it look novel.

I can illustrate this with a research project I just finished: MICRO. Over the last years, we have studied human music-related micromotion, the smallest actions it is possible to produce and perceive. This is new because no one has studied such motion in a musical context before. It is also unusual because the team comprised researchers from musicology, psychology, human movement science, and computer science.

The MICRO project can be considered novel. However, does that mean that everything we did in the project was novel? Some parts were, I guess. For example, we collected data by running the Norwegian Championship of Standstill annually. This was new and unusual the first time we did it. We even got quite a lot of media interest (it is not so often that music research is featured in the sports news on national TV).

However, collecting data once does not make for outstanding science. Research is about asking questions, finding answers, and verifying those answers. Repeating experiments, making slight modifications to the research design, improving the methods, refining the analyses. That is what solid research is about.

I have researched human music-related micromotion for nearly ten years now. We have some answers, but there are many open questions. Many of these questions are neither new nor unusual any longer. But if we want to understand more about what is actually going on inside our bodies when we experience music, we need to continue researching what is not any longer new and unusual. That is about doing solid research, not novel.

Open Research is better research

I believe that open research is better research. Opening the research process makes researchers think more carefully about what they do and how they document it. This takes (some) more time than working closed. But it also makes it easier for others to understand what has been done. This is important from a peer review perspective. It also facilitates incremental research.

The MICRO project has been an open research flagship project. I began by sharing the funding application openly. Throughout the project, we have continuously described how we have worked. The data has been released in the Oslo Standstill Database, and source code has been shared on GitHub. All of this has taken time “away” from publishing journal articles. However, it is time for researchers to publish fewer articles and focus more on making more data, code, etc., available.

Opening the research process is part of solidifying the research. As researchers, we cannot hide behind a “black box” any longer. Everyone can scrutinize what we have done. In fact, I hope that more people will analyze our data and develop our code. That is part of the incremental nature of science.

Summing up

I am not against novel research. However, I think we have gotten to a point where there is too much focus on novelty. If you are applying for a large research grant, it may make sense to doing something new. But it must be possible to submit a presentation to a conference or a manuscript to a journal based on plain, solid research. That may, in fact, be novel in itself! Hopefully, the transition to open research may actually help to focus more on solidity instead of novelty.

Open Research puzzle illustration

It is challenging to find good illustrations to use in presentations and papers. For that reason, I hope to help others by sharing some of the illustrations I have made myself. I will share them with a permissive license (CC-BY) to be easily reused for various purposes.

I start with the “puzzle” that I often use in presentations about Open Research. It outlines some of the various parts of the research process and how they can be made (more) open. I often think about the blocks as placed on a timeline from left to right.

An English-language version of my Open Research “puzzle.”

We will have a Norwegian conference on Open Research later this year, so I decided to make a version in Norwegian too for the conference page:

A Norwegian-language version of the Open Research “puzzle.”

Feel free to grab the images above. If you are interested in better versions, I have posted PDFs of both versions and the source presentation files (in PPTX and ODP) on GitHub. So head over there to download either the Norwegian or English files.

Why universities should care about employee web pages

Earlier this year, I wrote about my 23 tips to improve your web presence. Those tips were meant to encourage academics to care about how their employee web pages look at universities. Such pages look different from university to university. Still, in most places, they contain an image and some standard information on the top, followed by more or less structured information further down. For reference, this is an explanation of how my employee page is built up:

My employee page at UiO contains both standard and customized elements.

Arguments for why universities should care

Academics need to be visible online. If you don’t publish and disseminate your research, it won’t have an impact. So it is in our own interest to have up-to-date personal pages with information about what we do. I would argue that it is also in the interest of universities that their employee’s personal pages are up-to-date and look good. My argument goes like this:

  1. The people are the most important asset of a university
  2. The web is the most important dissemination channel
  3. Hence the employee pages should be the most important part of a university web page

For some reason, this appears to be a radical statement. In my experience, many universities think of the employee pages as a “phonebook.” It is a static page with minimal information about how to get in touch with the employee. There is often a template for the page, including information about educational background. Sometimes there is also information about courses taught and research output. But rarely does it contain much more stuff. There also seems to be little institutional interest in maintaining and improving such web pages.

The employee page is an important research infrastructure

Last year, I wrote in the UiO newspaper about considering the university’s web pages as a research infrastructure. In my experience, the web pages of a university are maintained by a communication department. They have one opinion on the purpose of the web pages: communication with students and the general public. I agree that these are the largest user groups of our web pages. But, in addition, we need to remember that researchers also communicate with other researchers.

Research communication is not the same as researcher communication. The former is dissemination activities targeting the broad public. The latter is based on ongoing intellectual exchange with research colleagues around the world. In my experience, communication departments care mostly about the first category. It is typically the role of university libraries to care about the other. Unfortunately, not many librarians are involved in the making and structuring of web pages. In my opinion, they should be.

Web pages as part of the transition to Open Research

I have previously written about why Open Research is better research and why I prefer Open Research over Open Science. In this context, I would mention that I believe university web pages, particularly employee web pages, are key to making a full transition to Open Research. Yes, we should focus on making publications Open Access and datasets FAIR. That should happen through proper repositories with unique IDs, and so on. However, the ambition of moving towards Science 2.0 goes beyond only opening the research results. We also need to open the various parts of the research process. Then I am thinking about the various building blocks of Open Research, as sketched in this figure:

The building blocks of an Open Research ecosystem.

The various parts in this ecosystem will live in different repositories and be scattered around the web. In my thinking, a person’s employee page is the place to gather all this information. It can serve as the hub of an academic’s activities.

Empowering the academics

It is in the interest of universities to provide their employees with the tools needed to store, share, and link to their research material. Many universities don’t seem to care too much about this. The result is that many employees don’t care either. Those who care will make their own solutions. Many set up private web pages with their own domains. Others use one of the social media sites for academics. These sites have understood how to make it fun to add information. It is a pity that universities don’t do the same.

At UiO, we are fortunate to have the possibility to edit the content of our personal pages. There are still things that could improve our system, and I regularly nag both the IT and communication departments about those issues. Still, I am fortunate to work at a university that empowers its academics with the possibility to update their own information. That is not the case in all universities. Some universities don’t allow employees to modify anything at all. I think that is a bad idea. It is bad for the employees, the university, and the transition to Open Research.

To all university leaders out there: how do you work with your university’s employee pages? To all academics: remember to update your personal page! And if you are not allowed to, ask your leaders to give you the tools and access to do so.

Soft drop shadows in LibreOffice Draw

My new book will be published Open Access, and I also aim only to use open-source tools as part of the writing process. The most challenging has been to figure out how to make nice-looking illustrations.

Parts of the book are based on the Ph.D. dissertation that I wrote a long time ago. I wrote that on a MacBook and made all the illustrations in OmniGraffle. While it was quite easy to make the switch to Ubuntu in general, OmniGraffle has been one of the few programs I have really missed in the Linux world. True, many graphics tools are available, but none with the same type of finesse as OmniGraffle.

After trying and failing with many of the most popular vector graphics tools, I have ended up using LibreOffice Draw to make all the illustrations. It is more limited than, say, Inkscape but provides the tools I need. It was also easy to get used to the workflow. The only thing I have struggled with figuring out is how to create nice-looking soft drop shadows. In OmniGraffle, the shadow function immediately looks nice, such as in this illustration of how one can think of an instrument as the mediator between action and sound:

My first attempts at adding a shadow in LibreOffice Draw ended up with a gruesome non-soft shadow:

I searched around, but could only find all sorts of complicated solutions. This was last year, and I decided to move on with the project without shadows. I could have moved to another graphics tool but had already spent quite some time making Draw figures. So I decided that shadows were not that important after all.

As part of the final manuscript completion, I am currently making minor changes to all the figures. Out of the blue, I found out that the latest version of Draw now has a more developed shadow section.

Hooray! By changing the “distance,” “glow,” and “transparency” settings, I now finally have soft drop shadows in place:

Draw does still not provide the same smooth user experience as OmniGraffle. But getting the soft drop shadow function in place helps a lot!