Fraunhofer Institute for Digital Media Technology IDMT
Fraunhofer Institute for Digital Media Technology IDMT

Automatic Music Analysis

Audio and Visual Content Analysis

Audio signal processing and machine learning for music analysis

Audio signal processing and machine learning are revolutionizing music analysis. From audio matching to music annotation and similarity search, from automatic music transcription to music generation, new application possibilities are emerging for broadcast monitoring, music search and recommendation, music production, and music learning programs.

Our goal is to enable quick and customized access to musical content through our approaches and techniques for automatic music analysis. We are developing practical solutions that are applicable in various domains such as the music industry, entertainment, education, and music production. In addition to enhancing existing technologies, we aim to showcase new application possibilities for automatic music analysis and contribute to the evolution of algorithms and methods.

News and upcoming events

 

Event / 17.3.2025

DAS I DAGA 2025

We are presenting our research at the 51st annual conference of the German Acoustical Society (DEGA). Visit us at booth B4-34.

DAS I DAGA 2025
 

Event / 11.3.2025

Data Technology Seminar 2025

Presentation “Cross-modal content analysis: finding, identifying and analyzing people in media” at the EBU event for innovators in AI, data and media technology.

DTS 2025
 

© Fraunhofer IDMT/bongkarn - stock.adobe.com

Press Release / 12.4.2024

Advertising monitoring for SWR radio programs

Our audio matching replaces manual checking of broadcast commercials

SWR Media Services
Expand all Close all

Research

Understanding Music

How do I quickly find a suitable music piece in a large music catalog? Can I automatically receive recommendations for the perfect beat that harmonizes well with a music production I'm currently working on? Which programs in my archive are the most successful? These are typical questions where our technologies for automatic music analysis can help.

Audio signal processing and machine learning have fundamentally changed music analysis. The multidisciplinary research field "Music Information Retrieval" encompasses algorithms and techniques for extracting musical information from audio data, transforming it into interpretable formats. The results are applied in areas such as broadcast monitoring, music search and recommendation, music production, content tracking, and music eductaion.

AI-based music analysis technologies

General challenges in automatic music analysis include processing large amounts of data, considering musical diversity and context, robustness to variations in recording quality, and the efficient deployment of real-time processing for various applications.

Audio-Matching

Audio matching via audio fingerprinting enables the identification of specific audio recordings in music collections and streams. Media content is compared and matched based on acoustic fingerprints. Audio matching is used for analyzing music usage in broadcast monitoring, content tracking applications, archive maintenance, as well as in music search engines and recommendation systems.
 

At Fraunhofer IDMT, we research how to further improve the accuracy and efficiency of audio matching techniques in order to enable more precise detection and identification of media content.

Annotation and similarity search for music

Annotation and similarity search for music facilitate the organization of music collections and simplify access to musical content. The use of metadata allows for versatile search and recommendation systems, automating the discovery of suitable music or musical elements. This is applicable, for example, in end-user streaming services or music production.


We are working on enhancing annotation and similarity search, particularly for large and diverse music collections, while also considering to user preferences and contextual information.

Automatic music transcription

Automatic music transcription involves converting music signals into symbolic music notation and extracting musical structures such as melodies, chords, and rhythms. These techniques are used in music learning programs, music game development, and music theoretical studies.


The specific challenges of automatic music transcription lie in precisely, reliably, and real-time capturing complex musical structures, even in polyphonic musical pieces or situations with background and ambient noise.

Automatic music generation

Automatic music generation involves the development of algorithms and AI systems capable of creating their own original musical pieces or parts thereof. It provides automated support in the music production process and during live performances, for instance, by generating melodies based on harmonies. This emerging field  introduces new creative approaches to music composition and production.


However, automatic music generation is still a relatively young research field and requires further progress to produce realistic and coherent musical results that meet the expectations of music creators and listeners. At Fraunhofer IDMT, we are researching ways to make the AI composition process transparent and controllable. Our aim is to support the creative collaboration between music creators and AI.