Thanks to the new Active Ambient Lighting, passengers can now experience music with three senses: they hear the music, they "feel" the music by means of integrated sound resonating transducers in the car seats, and, as of this year, they "see" the music thanks to the matching lighting control add-on by Fraunhofer IDMT. For this novel visual experience, the light strips are used which are installed in the instrument panel and in the front door panels.

Fraunhofer IDMT´s know-how for a new music experience in the car

For the development and technical implementation of the new interior feature, the German automaker relied on the Ilmenau research institute's years of expertise in the field of real-time processing of audio data. Fraunhofer IDMT is internationally renowned for its audio signal processing and music analysis know-how using Artificial Intelligence (AI) and has already been a research and development partner for various interactive music applications.

There were several challenges to overcome when implementing the sound-controlled light adaptation. For example, the spectral and dynamic characteristics of various audio signals – from speech to hip-hop, classical, smooth jazz, and heavy metal – had to be transformed into a beat-synchronized color visualization in real-time.

Time-synchronous music visualization

"Correlated visualization with low latency is enormously important for the synchronous experience of sound and light. This is quite crucial for the emotional effect", explains Holger Grossmann, principal researcher at Fraunhofer IDMT´s Semantic Music Technologies group.

With the new audio-driven light rendering technology, time-synchronous music visualization is made possible. For example, fast sequences of beats are visualized by means of rapid changes of light, and flowing rhythms tend to be depicted with softly merging light moods. 

Another challenge was that the sound-matched light rendering had to work for both loud and soft sounds, even when the differences between the sounds of a piece of music are very small.

"Thus, our intelligent signal analysis installed in the car has to adapt very flexibly to the respective audio data", says Holger Grossmann.

Balanced visualization of sounds

For safe use in the vehicle, it was also important to ensure a balanced visualization of the sounds so as not to distract the driver from what is happening on the road.

"We are proud that we can now tell another success story of our intelligent music analysis with the sound visualization for the new generation E-Class-cars, and we look forward to future next-generation collaborations”, says Holger Grossmann.