Visualizing some videos from the AIST Dance Video Database

Researchers from AIST have released an open database of dance videos, and I got very excited to try out some visualization methods on some of the files. This was also a good chance to test out some new functionality in the Musical Gestures Toolbox for Matlab that we are developing at RITMO. The AIST collection contains a number of videos. I selected one hip-hop dance video based on a very steady rhythmic pattern, and a contemporary dance video that is more fluid in both motion and music.

Hip-hop dance

The first I have looked at a couple of different files. Let us start with this one:

We can start by looking at the motion video from this. While a motion video gives less information about context, I often find them interesting to study since they reveal the essentials of what is going on.

And from the motion video we can look at the motiongrams and average image:

The horizontal motiongram reveals the repetitiveness of the dance motion, but also some of the variation throughout the different parts. I also really like the “bump” in the vertical motiongram. This is caused by the couple of side-steps he is doing midways in the session. The “line” that can be seen throughout the horizontal motiongram is cased by the cable in the back of the video.

Contemporary dance

And then I looked at another video, with a very different character:

From this we get the following motion video (wait a few seconds, since there is no dance in the beginning…):

The average image and motiongrams from this video reveal the spatial distribution of the dancer’s motion on stage. Here it is also possible to see an artifact of the compression algorithm of the video file in the beginning of the motiongrams.

I really look forwards to continue the explorations of this wonderful new and open database. Thanks to the AIST researchers for sharing!

Motiongram of high-speed violin bowing

I came across a high-speed recording of bowing on a violin string today, and thought it would be interesting to try to analyze it with the new version of the Musical Gestures Toolbox for Python. This is inspired by results from the creation of motiongrams of a high-speed guitar recording that I did some years ago.

Here is the original video:

From this I generated the following motion video:

And from this we get the following motiongram showing the vertical motion of the string (time running from left to right):

This motiongram shows the horizontal motion of the string (time running downwards):

Great example of a sound-producing action!

Podcast on Open Research

I was in Tromsø to hold a keynote lecture at the Munin conference a month ago, and was asked to contribute to a podcast they are running called Open Science Talk. Now it is out, and I am happy to share:

In this episode, we talk about Music Research, and how it is to practice open research within this field. Our guest is Alexander Jensenius, Associate Professor at the Department of Musicology Centre for Interdisciplinary Studies in Rhythm, Time and Motion (IMV) at the University of Oslo. He is also behind MusicLAb, an event-based project where data is collected, during a musical performance, and analyzed on the fly.

Thanks to Erik Lieungh and the rest of the team at the University Library at UIT The Arctic University of Norway. They are doing a great job in developing Open Science tools and strategies!

Keynote: Experimenting with Open Research Experiments

Yesterday I gave a keynote lecture at the Munin Conference on Scholarly Publishing in Tromsø. This is an annual conference that gathers librarians, research administrators and publishers, but also some researchers and students. It is my first time to the conference, and found it to be a very diverse, interesting and welcoming group of people.

A poster tweet from the Munin conference team.

Most of the other presenters talked about issues related to publishing academic texts, and with a particular focus on the transition to open access (OA). My presentation was focused on MusicLab, an open research pilot project we are running at the University of Oslo.

MusicLab is a collaboration between RITMO and the University Library, and it is a great example of how cool things can happen when progressive librarians work together with cutting-edge researchers. If you never heard about it before, here is a 42-second introduction to what MusicLab is all about:

Since lots of people talked about Open Science at the conference, I started out by arguing for why I believe that Open Research is a more inclusive term than Open Science. I then went on to identify some of the parts that people think about when talking about Open Research:

Some of the building blocks of an Open Research ecosystem.

As can be seen from the slide above, Open Access (which should probably be called Open Publication instead, since many people mistake it to mean Open Research) is just one part of the whole picture. In the picture above, I am also thinking about these building blocks as being placed on a “timeline” going from left to right, although there may certainly be recursive parts of the model as well.

As a researcher, the publication part is typically happening fairly late in the process, so I always try to remind people that the actual research happens before it is published. For example, the writing process is also something that should be thought of as open process, I think, I mentioned some of my explorations into using various tools for writing Open Manuscripts:

None of these are perfect, however, and for some upcoming projects I am thinking about exploring Authorea and Jupyter Notebook as writing tools. After my talk I also got a recommendation for Bookdown, which I would like to look more at as well (although I have for a long time avoided getting into R, since I am currently investing some time in moving my code from Matlab to Python).

MusicLab

After the fairly long introduction, I finally got to the main point of the talk, which is that of MusicLab. Here are some of the slides from that part:

A MusicLab event is built around a concert, but also typically contains a workshop, panel discussion, data collection, and data jockeying.
Some photos from MusicLab vol. 1, which was focused on muscles, and with a performance by Marco Donnarumma (Photos: Simen Kjellin, UiO).
The MusicLab events are part of a pilot project which is aimed at discovering new ways of doing research, education, and dissemination in open ways.

Challenges

One of the points of MusicLab is to jump in and do something that everyone says is “impossible”… We have of course, have our set of challenges, and particularly related to:

  • Privacy (GDPR)
  • Copyright and licenses
  • Storage
  • Archive

I will write more about all of these later, but here just some slides to summarize some points:

Dividing the people at a MusicLab into three groups, helps when it comes to identifying and solving issues of privacy.
We have not solved the problem of copyright in relation to Open Research yet, but we start to get an overview of all the problems…
Storage is not only about saving files somewhere. They need to be usable as well, ideally right away.
This is the list of files from MusicLab vol. 4, and some of the tools we want to use to analyze them.

We have more challenges than solutions at the moment. But it is good to see that things are moving in the right direction. The dream scenario would be a combination of the multimedia visualization tools from Repovizz combined with the interconnectivity of Trompa, the CC-spirit of Audio Commons, the versioning of GitHub, the accessibility and community of Wikipedia, and the longterm archiving of Zenodo. While that may sound entirely far-fetched right now, it could be a reality with some more interoperability.

I got lots of interesting feedback after my talk. It was particularly interesting to hear several people commenting on the importance of having more people from the arts and humanities involved in discussions about Open Research. I am happy to be one such voice, and hopefully MusicLab can inspire others to push the boundaries for what is currently possible.

If you want to watch the entire thing, it can be found towards the end of this recorded live stream:

Some tips and tricks when writing academic papers

I have been teaching the course Research Methods, Tools and Issues in our MCT programme this semester. The last class was an “open clinic” in which I answered questions about academic writing. Here is a summary of some of the things I answered, which may hopefully also be useful for others.

Formatting

Your academic exam paper is not the place to experiment with fancy layout and formatting. Some basic tips:

  • Template: Choose a conservative template (but not too old-school). Check that it is in A4, as templates using a North-American “letter” paper size looks weird when/if printed in A4.
  • Font: A serif font typically looks more serious than a sans-serif, so “Times New Roman” or something similar is the safest choice.
  • Paragraphs: There are different ways of formatting paragraphs. The two most common ones are: (1) indented first lines, (2) spaces between lines. The first type is the most common in professional type-setting and is what you see in books and academic journals. It is also the most space-conservative. Making spaces between lines is what most people do when they write on computers. Choose whichever type you want, but do not mix the two.

Writing

It is impossible to cover everything about academic writing here. But these are the things I usually comment on when I supervise:

  • Long sentences: It is better to write short and meaningful sentences than long and complex ones. Many students think that their text will look more academic if they make long sentences. The truth, however, is that it is much more pleasurable to read well-formed and meaningful sentences. The general rule of thumb is to include only one point in a sentence. If in doubt, start a new sentence. It also helps to read your text out loud. Whenever you feel that
  • Short sentences: Some students only write very short sentences. This is not good either, so find a balance.

Spelling and grammar

Whatever you do, ensure that your texts are not full of spelling mistakes. You should also reduce the number of grammatical errors. There are so many tools available to help you these days, so there is no excuse for not using them before you submit your final text.

Figures

Figures are nice, please include them! But when you do, always think about this:

  • Label: Figures should always have a figure name (“Figure 1”) followed by an explanation of what the figure is about (“This figure shows…”). Ideally, the figure text should be sufficient to understand what the figure aims at conveying of information. Many people (like myself) like to browse through papers and books quickly, and use the figures as a way to quickly navigate the content. Then it helps if the figure texts are self-explanatory.
  • Text size: Figures are often made in different software than where you write. This means that you typically do not have full control of their size in the final layout, hence the text inside of the figure may be too large and too big. As part of the final layout, you should try to make the text size similar to the text size of the main document within which the figure is placed.
  • Units, labels and legends: If you include graphs or other types of representations of numbers, it is critical to include information about what the axes mean and the units that you have used (“Time (s)”, “Number of people”, “Vertical position (mm)”). You should also have clearly marked legends (if relevant) to explain what the different lines in your figure are.
  • Simplify: You should always aim to remove unnecessary stuff from figures so that the most important things are what people see. This follows Tufte’s ideal of aspiring for a high “data-ink ratio”.

References

Adding references to your text, and including a bibliography at the end of your document, is the clearest sign that you are writing an academic paper.

  • Consistency: Ensure that all citations have an entry in the bibliography. Similarly, all entries in the bibliography should be referenced in the text.
  • Reference manager: Use a reference manager to keep track of everything. While it is not perfect, I generally recommend Zotero. It works on all platforms, has an online front-end, and integrates with many writing platforms.

Submission

  • PDF: If you are not asked to do otherwise, always submit a PDF file. This will ensure that both content and layout are preserved for the final reader. Submitting your “raw files” (.docx, .pages, .odt, etc.) is problematic for a number of reasons. First, they may not be readable by people on different platforms (.pages files only work on OSX, for example). Second, often such raw files contain the history of the file, which you may not want the end reader to see. This may be particularly important if you have been using track changes.
  • Good naming: Always give your file a useful name. If your exam is anonymous, include your candidate number in the file name. If not anonymous, include your last name. Your examiner will probably download a zip-folder with all submissions. Having a bunch of files with names such as “exam.pdf”, “submission.pdf”, etc., is annoying.
  • Supplementary files: It is often fine/useful/required to submit supplementary material. Then it is usually good to have a list at the end of your main document describing what you have chosen to include (for example, a list describing audio files). If you have many supplementary files, you should zip them down and give them a useful name. Again: remember that the reader will download your submission together with a bunch of other things. It is your job to make the read as pleasurable as possible.

General form

The standard “IMRAD” form of a paper looks like this:

  • Abstract
  • Introduction
    • Motivation (could include a rhetorical question/something catchy)
    • Research question(s)
    • (Hypotheses)
    • Definitions
    • Limitations / scope
    • Overview of the paper
  • Background (either chronological or topical)
  • Methods (be precise – explain what you did, how, etc)
  • Results
  • Discussion
  • Conclusion

Many interaction papers, and also “NIME-like” papers, have a form something like:

  • Abstract
  • Introduction
  • Background
  • Method
  • Design
  • Implementation
  • Evaluation / Discussion
  • Conclusion