In production environments today, cars, airplanes, optical systems, medical devices and other products are manufactured in large individual components and then glued or fitted together. »Robot arms do the fitting work. Component tolerances play an important role: if they are too large, they can cause collisions and displacements,« explains Danilo Hollosi, Head of Acoustic Event Detection at Fraunhofer IDMT in Oldenburg. The error is often noticed too late, which leads to unscheduled downtime and high costs.

Non-contact acoustic process monitoring for plug connections

To avoid this situation and promptly detect faulty technology, Hollosi and his team have developed smart sensors that can be fitted directly to the machine or equipment and identify faults immediately. Crucially, the sensors are sensitive to airborne sound and recognize faults based on noises. »When mating connectors join together, it makes a click that the microphone or sensor picks up. If the click doesn’t happen, the acoustic monitoring system displays an error, which is reliably documented. At the same time, the relevant worker is notified«, explains the engineer. In automated manufacturing, the metadata is used for process documentation and quality assurance. A unique feature of the solution is that the maintenance system can distinguish between countless types of clicks and mechanical impacts, while also filtering out noise interference in loud production environments. »Not all clicks are the same. A mechanical connector sounds different to a light switch, a stapler or a biro. You’d be amazed how many different kinds of noise things make when they snap into position«, says Hollosi.

Sense of hearing for machines

Specially developed AI-based algorithms for audio analysis detect the interference and the target noises. Processing of the data is performed directly at the sensor. The complete condition monitoring system takes up little space: the microphone, audio signal processing technology, software and battery are currently housed in a casing no larger than a pack of cigarettes. But they can be made even smaller as well. The miniaturized solution is available in three variants and can be easily integrated into existing systems. Users also have the option of attaching the intelligent maintenance system to robots or of installing it several meters away from the machinery or equipment and at strategically valuable measuring points. In addition, the acoustic solution is scalable – an important condition for use in industrial applications.

»In effect, we give machines a sense of hearing for quality assurance. This allows manufacturers to punctually recognize signs of damage at an early stage, to reduce unscheduled downtime, to harmonize shop floor workflows and to increase the effectiveness of the overall plant«, summarizes the researcher.

Hearing, speech and audio technology HSA at the Fraunhofer Institute for Digital Media Technology IDMT in Oldenburg

The objective of the Hearing, Speech and Audio Technology Division (HSA) of the Fraunhofer Institute for Digital Media Technology IDMT is to transpose scientific findings related to hearing perception and human-technology interaction into technological applications. Its applied research priorities are the enhancement of sound quality and speech intelligibility, personalized audio reproduction and acoustic speech recognition and event detection with the help of artificial intelligence. A further focus is the use of mobile neurotechnologies, which facilitate the recording of brain activity and utilization of the resulting data outside the laboratory too. Application fields include consumer electronics, transport, the automotive sector, industrial production, security, telecommunications and healthcare. Through scientific partnerships, Fraunhofer IDMT-HSA has close links with the Carl von Ossietzky University of Oldenburg, Jade University and other institutions engaged in hearing research in Oldenburg. Fraunhofer IDMT-HSA is a partner in the »Hearing4all« Cluster of Excellence.

The Division Hearing, Speech and Audio Technology HSA is funded in the program »Vorab« by the Lower Saxony Ministry of Science and Culture (MWK) and the Volkswagen Foundation for its further development.