Professional language skills are an important basis for effective communication and successful teamwork. This is especially challenging for groups with different native languages and a variety of cultural backgrounds. In addition to specialist knowledge and language proficiency, other soft skills are in demand, such as active listening, experience with handling stress, a solution-oriented approach, decision-making ability and an affinity for transparent communication. To systematically detect language barriers and obstacles to personal interaction in everyday professional life, an automatic procedure for assessing language and communication skills has been developed within the »AudioCAT« research project. The automatic system is intended to help prospective nursing staff and doctors to demonstrate their own professional language skills in realistic situations.

Effective recruiting thanks to automated dialogue system

The aim of the technology is to make recruiting, in-house filling of vacancies and team building more efficient in the future, especially in the HR departments of organisations such as hospital associations or care facilities. Specifically, after further adjustments to the system by means of automated test procedures, applicants will be invited to solve job-related tasks. The important point here is that the test results are evaluated in real time in the background, and in this way the system’s AI can react to the participants’ actual skills during the ongoing process. For particularly realistic training, participants communicate verbally with the computer’s dialogue system in both natural everyday as well as technical language.

Automatic speech recognition and AI

What makes this communication possible is the automated speech recognition technology developed by Fraunhofer IDMT in Oldenburg, which has been integrated into the application. It can recognise spoken words and sentences and output them as text. With the help of the transcribed text, answers are analysed, classified and evaluated on the basis of machine learning methods.

Checking the test persons’ comprehensibility and communication level as realistically as possible is an important part of the process. In addition to the expected, profession-related answers, metrics from linguistics and language research are also incorporated. By using the server services of the Assistive Speech and Language Analysis Group at Fraunhofer IDMT in combination with algorithms and metrics for speech and language evaluation, it is possible to analyse pauses in speech, speaking rate or emphatic structure, among other things. This also allows statements to be made about language skills.

Digital tool for competence assessment in the medical sector

The test scenarios are created via a graphical editor in which not only consecutive questions but also expected answers can be defined. In this way, it is possible to react directly to the users’ spoken answers and to pose a logical next question. Here, »AudioCAT« also makes it possible for the questions to be read aloud in order to simulate a natural dialogue.

The technical development structure builds on evaluation by means of automatic speech recognition via Natural Language Processing (NLP) and Natural Language Understanding (NLU) as well as other data analysis methods. The »AudioCAT« project works with an exclusive software programmed by worldiety GmbH. The aim in future is to support staff evaluation in the long term by providing a competence assessment tool for the medical sector. The simulated dialogue with the system can already be tested on the website: www.audiocat.de

Even though the funding phase has ended, the project partners are continuing to optimise the AudioCAT system. Your voice can help us to bring the system closer to realistic scenarios and to improve speech interaction with humans. Register now for the free version and experience our language competence tool on the following website: https://audiocat.de/exam.  

Hearing, Speech and Audio Technology HSA at Fraunhofer IDMT in Oldenburg

Founded in 2008 by Prof. Dr. Dr. Birger Kollmeier and Dr. Jens-E. Appell, the Fraunhofer Institute for Digital Media Technology IDMT’s Branch for Hearing, Speech and Audio Technology HSA stands for market-oriented research and development with a focus on the following areas:

• Speech and event recognition
• Sound quality and speech intelligibility
• Mobile neurotechnology and systems for networked healthcare

With in-house expertise in the development of hardware and software systems for audio system technology and signal enhancement, over 100 employees at the Oldenburg site are responsible for transferring scientific findings into practical, customer-oriented solutions.

Through scientific cooperation, the institute is closely linked to the Carl von Ossietzky University, Jade University of Applied Sciences, and the University of Applied Sciences Emden/Leer. Fraunhofer IDMT is a partner in the »Hearing4all« cluster of excellence.

The Oldenburg Branch for Hearing, Speech and Audio Technology HSA is funded in the program »Vorab« by the Lower Saxony Ministry of Science and Culture (MWK) and the Volkswagen Foundation for its further development.