According to the German Neurological Society, about 35% of stroke patients suffer from neurogenic speech and language disorders in the acute phase and 20% in the chronic phase. These disorders, known as aphasia and dysarthrophonia, can occur in isolation or in combination. By means of hybrid and interactive teletherapeutic approaches, a software-based learning system is being developed within »HiSSS« that can be used in both video and in-person therapy. In addition, patients can use it for their own personal training without the need for supervision. »The system is mobile, flexible and works without permanent contact to a therapist. This allows patients to train more independently and in a fun way through integrated gamification elements, without interrupting their therapy or limiting quality«, explains Klaus Kugelmann, CEO of SpeechCare GmbH. 

»We’re pleased that our hybrid approach enables us to further introduce the advantages of speech technologies into therapeutic applications«, reports Laura Tuschen, Head of Assistive Speech and Language Analysis at Fraunhofer IDMT’s Oldenburg Branch for Hearing, Speech and Audio Technology HSA. »This opens up completely new possibilities for participation and communication for those affected.« Since the beginning of the year, the Therapeutic Product Guidelines (in German) have smoothed the way for the use of video-assisted therapy, which the consortium hopes will give the planned technologies a further boost.

Supporting the recovery of verbal communication by means of machine learning techniques

Together with colleagues from the »Automatic Speech Recognition« group at Fraunhofer IDMT, Laura Tuschen’s team is supporting the project by collecting speech and language data. Here, machine learning methods, automatic speech recognition and assessment as well as optical face and facial expression analysis are used to generate direct and individual feedback during training in the app. Collaboration between the different research areas at Fraunhofer IDMT and within the consortium reflect the project’s interdisciplinarity. »HiSSS« is funded by Germany’s Federal Ministry of Education and Research under the aegis of VDI/VDE Innovation + Technology GmbH and will run until 2025.

The »Assistive Speech and Language Analysis« group led by Laura Tuschen was established last year and focuses on the development and utilisation of digital speech processing technologies. To this end, the experts use machine learning and computational linguistics to explore automatic speech, language, and voice analysis. In the area of health research, Fraunhofer IDMT works closely with the University of Oldenburg. For several years now, the Oldenburg Branch for Hearing, Speech and Audio Technology HSA has been conducting research in the field of »Connected Health«. All research and development activities related to health foreground the flexible use of healthcare systems without clinical infrastructure.

Further information on the website of the German Neurological Society (in German).

Hearing, Speech and Audio Technology HSA at Fraunhofer IDMT in Oldenburg

Founded in 2008 by Prof. Dr. Dr. Birger Kollmeier and Dr. Jens-E. Appell, the Fraunhofer Institute for Digital Media Technology IDMT’s Branch for Hearing, Speech and Audio Technology HSA stands for market-oriented research and development with a focus on the following areas:

• Speech and event recognition
• Sound quality and speech intelligibility
• Mobile neurotechnology and systems for networked healthcare

With in-house expertise in the development of hardware and software systems for audio system technology and signal enhancement, over 90 employees at the Oldenburg site are responsible for transferring scientific findings into practical, customer-oriented solutions.

Through scientific cooperation, the institute is closely linked to the Carl von Ossietzky University, Jade University of Applied Sciences, and the University of Applied Sciences Emden/Leer. Fraunhofer IDMT is a partner in the »Hearing4all« cluster of excellence.