The MIRAGE project aims to develop a ground-breaking AI system for music analysis to enhance computers' ability to listen to and understand music.
This provides new ways to gain deeper insight into the language of music using cutting-edge technology, ultimately helping human listeners better understand and appreciate music.
The MIRAGE project is funded by the Research Council of Norway under the program IKTPLUSS, and is led by Olivier Lartillot at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion at the University of Oslo (UiO).
Enhancing music accessibility and engagement through AI-driven analysis
Computer-supported analysis tools extract large amounts of information about musical elements from the music such as timbre, pitch, rhythm, tonality, and form. An important application of the technology is to make music more accessible and engaging, and it can also be used in music cognition and music therapy. The AI technology is designed as a modular and hybrid architecture that combines both machine learning and symbolic AI models related to musicology and music cognition.
Machine learning is specifically utilised for a crucial initial step in music analysis, which involves music transcription. The goal is to detect all the notes played in the music from an audio recording, and accurately measure their timing and pitch height to create a representation similar to sheet music. To achieve this, researchers Lars Monstad and Anders Elowsson have been experimenting with various models. They started with a home-made framework based on the concept of "deep layered learning," which uses intermediate machine learning targets to reveal the musical organisation. Additionally, the researchers have integrated the latest state-of-the-art advances in deep learning, specifically utilising Convolutional Neural Networks and a Hierarchical Frequency-Time Transformer model.
"Sigma2/NRIS was instrumental in our training process, meeting our significant need for processing power.
Their well-documented user pages facilitated the creation of an efficient training environment and the scheduling of large tasks, making it an invaluable resource in our project."
Lars Monstad, Researcher at UiO
Bridging the past and present
The MIRAGE project has a specific emphasis on transcribing old archives of Norwegian folk music tunes, with a particular focus on Hardanger fiddle music. The aim is to make this music more accessible to the public. To ensure the collection of high-quality training data, we enlisted the expertise of renowned musicians, including Olav Luksengård Mjelva. We asked them to record tunes and provide precise annotations of all the notes they played using an annotation interface developed specifically for this purpose.
In addition to Norwegian folk music, our technology has also played a crucial role in expediting the composition process for various Norwegian television shows. Notably, composer Peter Baden has effectively utilised this tool to enhance his workflow, thereby demonstrating the demand for innovative tools in the professional music industry. Furthermore, the ongoing development of these tools is being investigated with the support of the UiO Growth House at the University of Oslo.