The research activities in the field Industrial Sound Analysis are focused on the development of AI-based methods for automated acoustic monitoring of products and processes for use in in-line and end-of-line quality control, process monitoring and predictive maintenance.
Your car is a good example to show the potential of quality assessment based on acoustic signals. You know the familiar driving sound caused by the engine, tyre wear and airflow. But what do you feel when this noise changes? A rattling or grinding noise might worry you and make you wonder if your car is still safe to drive. You probably react to this unknown and unexpected condition by driving to the workshop or at least stopping your car. It's no different in industry. We analyse the sound of your product or production process and let you know when something is not working as expected.
In industry, wherever there is movement, there are audible sounds that indicate the quality of products or processes. At the Fraunhofer IDMT in Ilmenau, Germany, we develop AI-based methods for sound analysis - especially for the analysis of industrial sounds - and thus create innovative approaches for automated acoustic monitoring (amo) of products and processes. amo can be used along the entire value chain for quality assessment and offers added value where, for example, optical monitoring methods reach their limits.
Our team of scientists from the fields of data science, data analysis, software development and project management is therefore researching AI-based solutions for audio signal analysis. The innovation at amo is that the measurement data is processed without any connection to an external cloud, so our solutions can be used locally within the company or directly on the machine.
Machine and plant manufacturers are faced with the challenge of providing their customers with comprehensive sensor-based machine monitoring. However, existing methods cannot solve all the problems that arise, such as unexpected downtime, continuous monitoring of the machine or evaluation of existing machine data. In addition, there is a shortage of specialists who know their machines so well that they can immediately identify faults by sound and take action if necessary.
Research field "amo”
This is a modal window.
Reference project with Procter & Gamble
We have developed AI algorithms to analyze the sound of home appliances for monitoring their condition with over 97% accuracy. This enables the use of interactive apps to track the status of household activities and the involved device leading to improved user guidance and higher customer satisfaction.
In this experiment, three different operating states were simulated: running, broken, and running under heavy load. Each motor is in one of these operational states which sound differently. The researchers placed a MEMS (micro-electro-mechanical systems for signal recording) microphone in front of the engines, which recorded the operating sounds of the running engines. Recordings were made for each engine, which were analyzed using machine learning techniques with and without background noise.
This is a modal window.
The principle demonstrator "Air-Hockey table" offers visitors to trade fairs and conferences a sporty variety in addition to a business talk.
Using an Air-Hockey table modified for research purposes, the researchers are working on novel approaches for acoustic quality assurance in an industrial context. They use pucks, which are made of different materials and cause different "pling" sounds as soon as they hit the board of the game table. During the game, these acoustic signals occur so frequently and irregularly that they can be used for analysis by means of machine learning methods to make a reliable statement about the material the pucks are made of.
This non-contact method, in which Fraunhofer IDMT combines its many years of expertise in the fields of acoustic measurement technology, signal processing and machine learning, can be used, for example, to detect material defects or in the in-line monitoring of welding processes. If acoustically perceptible defects are already detected during the process, it can be aborted and restarted promptly. Fraunhofer IDMT's acoustic testing method is also non-destructive and thus serves to reduce expensive test scrap.
This is a modal window.
The marble track demonstrator illustrates the performance of the acoustic monitoring method in industrial applications. The detection of defects in surfaces was chosen as an application example.
Three differently coated balls roll in random order through the ball track system. The moving noises of the balls are picked up by small microphones and analyzed by machine learning algorithms. In this way, the balls are monitored in real time and identified by their specific rolling sound. The analysis result - the order of arrival - is immediately shown graphically on a display. With this principle demonstrator, the institute shows new acoustic methods for quality assurance - contactless, non-destructive and safely integrated.
This is a modal window.
AI demonstrator for a smart acoustic burglary surveillance system.
This is a modal window.
Reliably and automatically detect audible leaks thanks to airborne sound analysis and machine learning methods.
Compressed air is an indispensable resource for many German industrial and trade companies for the operation of machines and plants. At the same time, it also represents a high cost factor on the electricity bill. On average, companies waste 30 percent of the energy generated due to the unnoticed escape of expensive compressed air. To detect such leaks, they rely, among other things, on ultrasound testing methods and on the mere hearing of trained personnel.
Fraunhofer IDMT has now tested in an experiment whether this "hearing" can be automated and reproduced using microphones in combination with machine learning methods in order to develop a reliable system for leakage detection. First results show that this is generally possible.
This is a modal window.
Reliably detect hairline cracks in turned parts with airborne sound analysis and AI
As part of the quality inspection, the turned parts shown usually undergo a 100 percent visual final inspection. They are checked for finest hairline cracks, which are hardly visible, but indicate a quality defect. In an experiment, Fraunhofer IDMT investigated to what extent hairline cracks can be detected automatically by airborne sound analysis with AI. As a testing step in the final inspection, the impact of a turned part on a plate was recorded and assigned to the classes "ok" and "not ok" accordingly. The scientists investigated whether the sound spectra of the impact allow a distinction between good and bad parts, which parameters influence these sound spectra (e.g. drop height or material of the plate) and which microphone positions are best suited for the recordings as well as for later analysis. First results showed that it is generally possible to distinguish turned parts into two classes based on the impact sound.
This is a modal window.
Turn up your speakers: "We save Christmas"
We offer services in the field of research and development and point out numerous possible applications of acoustic monitoring (amo) to our customers in the context of an industrial project. In addition, we work in partly publicly funded research and development projects to develop application-specific solutions together with wide-ranging expert consortia.
The applications for acoustic monitoring are many and varied. Wherever audible sounds occur in processes or on products, the use of automated acoustic testing methods can add value. We are continually exploring the use of our AI algorithms for acoustic quality control in welding and machining operations, as well as other applications where acoustic signals provide information about quality and grade.
This is a modal window.
Acoustic monitoring can be used in machining or welding, among other applications. Use cases are: Inline monitoring, process monitoring or predictive maintenance.
Our scientists exchange information within their research community and with stakeholders from industry and present current research results at scientific meetings and conferences. In addition, we regularly publish articles in professional journals that deal with application topics in our research field, including the German speaking magazines "Maschinenmarkt" or "Fertigungstechnik".
| Jahr Year | Titel/Autor:in Title/Author | Publikationstyp Publication Type |
|---|---|---|
| 2024 | Multimodale Fehlererkennung in Laserablationsprozessen Talagini Ashoka, Anitha Bhat; Anding, Katharina |
Zeitschriftenaufsatz Journal Article |
| 2024 | Towards Measuring and Forecasting Noise Exposure at the VELTINS-Arena in Gelsenkirchen, Germany Ngamthipwatthana, Pitchapa; Götze, Marco; Kátai, András; Abeßer, Jakob |
Konferenzbeitrag Conference Paper |
| 2024 | DOL3: Distilled OpenL3 audio embeddings for lightweight audio classification Kehling, Christian; Gourishetti, Saichand |
Konferenzbeitrag Conference Paper |
| 2024 | Acoustic insights into the corn extrusion process for enhanced quality control Gourishetti, Saichand; Chauhan, Jaydeep; Krüger, Tanja; Grollmisch, Sascha; Liebetrau, Judith; Bös, Joachim |
Konferenzbeitrag Conference Paper |
| 2024 | Selbstüberwachtes Vortraining zur Verbesserung automatischer Audioklassifikationsalgorithmen Grollmisch, Sascha; Abeßer, Jakob; Bös, Joachim |
Konferenzbeitrag Conference Paper |
| 2024 | Aktuelle Forschungsschwerpunkte in der akustischen Ereignisdetektion Abeßer, Jakob; Grollmisch, Sascha; Bös, Joachim |
Konferenzbeitrag Conference Paper |
| 2023 | Empirical Study on DED-Arc Welding Quality Inspection Using Airborne Sound Analysis Chauhan, Jaydeep; Gourishetti, Saichand; Rohe, Maximilian; Sennewald, Martin; Hildebrand, Jörg; Bergmann, Jean Pierre |
Konferenzbeitrag Conference Paper |
| 2023 | Temporal Resolution of Acoustic Process Emissions for Monitoring Joint Gap Formation in Laser Beam Butt Welding Kodera, Sayako; Schmidt, Leander; Römer, Florian; Schricker, Klaus; Gourishetti, Saichand; Böttger, David; Krüger, Tanja; Kátai, András; Straß, Benjamin; Wolter, Bernd; Bergmann, Jean Pierre |
Zeitschriftenaufsatz Journal Article |
| 2023 | Ohren für Schweißroboter Breitbarth, Kati; Krüger, Tanja; Liebetrau, Judith |
Zeitschriftenaufsatz Journal Article |
| 2023 | Automated Quality Inspection in Additive Manufacturing for Lightweight Construction: A New Approach Based on Virtual Sonic Data and Machine Learning (ML-S-LeAF) Yildiz, Ömer Faruk; Fritz, Alexander; Storch, Julian; Kátai, András; Ribecky, Sebastian; Hofmann, Peter; Talagini Ashoka, Anitha Bhat; Fassbender, Rene; Marckmann, Hannes; Grollmisch, Sascha; Jansen, Stefan; Adams, Christian; Kroh, Irina; Zaleski, Olgierd; Manohar, Aswin; Keuchel, Sören; Schröder, Thorben; Ren, Yaxiong; Boni, Christiano de; Balestra, Italo; Bös, Joachim; Ferretti, Raphael; Schötz, Johannes; Merschroth, Holger; Gross, Peter; Weigold, Matthias |
Konferenzbeitrag Conference Paper |
| 2023 | Arc Welding Process Monitoring Using Neural Networks and Audio Signal Analysis Gourishetti, Saichand; Chauhan, Jaydeep; Grollmisch, Sascha; Rohe, Maximilian; Sennewald, Martin; Hildebrand, Jörg; Bergmann, Jean Pierre |
Konferenzbeitrag Conference Paper |
| 2023 | Acoustic data acquisition for quality monitoring during Powder Bed Fusion with Laser Beam (PBF-LB) Ren, Yaxiong; Adams, Christian; Gross, Peter; Talagini Ashoka, Anitha Bhat; Kátai, András; Weigold, Matthias; Melz, Tobias |
Konferenzbeitrag Conference Paper |
| 2023 | Monitoring of Joint Gap Formation in Laser Beam Butt Welding Using Neural Network-Based Acoustic Emission Analysis Gourishetti, Saichand; Schmidt, Leander; Römer, Florian; Schricker, Klaus; Kodera, Sayako; Böttger, David; Krüger, Tanja; Kátai, András; Bös, Joachim; Straß, Benjamin; Wolter, Bernd; Bergmann, Jean Pierre |
Zeitschriftenaufsatz Journal Article |
| 2022 | Potentials and Challenges of AI-based Audio Analysis in Industrial Sound Analysis Gourishetti, Saichand; Grollmisch, Sascha; Abeßer, Jakob; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2022 | Intelligentes akustisches Monitoring durch ausgewählte Mikrofonierungskonzepte Fritsch, Tobias; Bös, Joachim; Grollmisch, Sascha; Gourishetti, Saichand; Hofmann, Peter; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2021 | Improving Semi-Supervised Learning for Audio Classification with FixMatch Grollmisch, Sascha; Cano, Estefanía |
Zeitschriftenaufsatz Journal Article |
| 2021 | The Sounds of Partial Discharge Gourishetti, Saichand; Werner, Sara; Kátai, András; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2021 | Investigating the influence of microphone mismatch for acoustic traffic monitoring Gourishetti, Saichand; Abeßer, Jakob; Grollmisch, Sascha; Kátai, András; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2021 | Investigation of the directional characteristics of the emitted airborne sound by Friction Stir Welding for online process monitoring Bös, J.; Katai, A.; Grätzel, M.; Other, S.; Stoll, B.; Rohe, M.; Hasieber, M.; Löhn, T.; Hildebrand, J.; Bergmann, J.P.; Breitbarth, K. |
Poster |
| 2021 | Partial discharge monitoring using deep neural networks with acoustic emission Gourishetti, Saichand; Johnson, David; Werner, Sara; Kátai, András; Holstein, Peter |
Konferenzbeitrag Conference Paper |
| 2021 | IDMT-Traffic: An Open Benchmark Dataset for Acoustic Traffic Monitoring Research Abeßer, Jakob; Gourishetti, Saichand; Kátai, András; Clauß, Tobias; Sharma, Prachi; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2020 | Techniques improving the robustness of deep learning models for industrial sound analysis Grollmisch, S.; Johnson, D.S. |
Konferenzbeitrag Conference Paper |
| 2020 | IAEO3 - Combining OpenL3 Embeddings and Interpolation Autoencoder for Anomalous Sound Detection Grollmisch, Sascha; Johnson, David; Abeßer, Jakob; Lukashevich, Hanna |
Vortrag Presentation |
| 2020 | Visualizing Neural Network Decisions for Industrial Sound Analysis Grollmisch, Sascha; Johnson, David; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2020 | Compressed air leakage detection using acoustic emissions with neural networks Johnson, D.; Kirner, J.; Grollmisch, Sascha; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2020 | Plastic Material Classification using Neural Network based Audio Signal Analysis Grollmisch, Sascha; Johnson, David; Krüger, Tobias; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2019 | Anwendung von (Luft-)Schallanalyse als ein Verfahren der berührungslosen Qualitätssicherung für die vorausschauende Wartung Kepplinger, Sara; Helbig, Mareike; Clauss, Tobias; Lukashevich, Hanna |
Konferenzbeitrag Conference Paper |
| 2018 | Störschallunterdrückung bei Luftschallanalysen in industriellen Fertigungsstrecken Nowak, Johannes; Grollmisch, Sascha; Cano, Estefanía; Lukashevich, Hanna; Liebetrau, Judith |
Konferenzbeitrag Conference Paper |
| 2018 | Stadtlärm - a distributed system for noise level measurement and noise source identification in a smart city environment Clauß, Tobias; Abeßer, Jakob; Lukashevich, Hanna; Gräfe, Robert; Häuser, Franz; Kühn, Christian; Sporer, Thomas |
Konferenzbeitrag Conference Paper |
| 2018 | Luftschallbasierte Rissdetektion von Metallteilen Liebetrau, Judith; Grollmisch, Sascha; Nowak, Johannes |
Konferenzbeitrag Conference Paper |
Fraunhofer Institute for Digital Media Technology IDMT