Fraunhofer Institute for Digital Media Technology IDMT
Fraunhofer Institute for Digital Media Technology IDMT

Facial Analysis Technologies

AI-Based Facial Recognition and Analysis

The face is the most important external characteristic of a person. Humans distinguish familiar faces from unfamiliar ones based on a unique combination of features that help us distinguish one individual from another or gauge the emotional state of another person. Fraunhofer IDMT's Facial Analysis Technologies provide a deep learning-based tool-set to automate facial recognition and analysis at scale. Our technologies enable a wide range of facial features to be extracted from image and video recordings, e.g. to identify characteristics such as age or gender. This automatically extracted information can be used in a wide range of commercial, industrial and institutional applications. Fraunhofer IDMT’s Facial Analysis Technologies combine state-of-the-art machine learning tools, which are combined, optimized and, if necessary, individually-trained depending on the application, for accurate facial recognition. 

Our solutions can be used in a wide range of applications. Examples and use cases include:

  • Automating the detection and recognition of actors and actresses in videos or photos for a variety of commercial media and visual archive applications.
  • Detecting correctly worn face masks for public health purposes.
  • Identifying animal life, such as great apes, for biodiversity conservation and environmental management projects.
  • Recognizing specific persons in image and video data to resolve virtual identities back to real ones for law enforcement purposes. 

By combining different technologies we enable the implementation of new solutions for a wide range of use cases. For example, Facial Analysis Technologies may be combined with other AI-based technologies such as visual object recognition, visual scene detection and classification, audio event detection and audio scene classification. Such combinations serve to deliver extended metadata to provide deeper analysis solutions for more complex tasks.

All of Fraunhofer IDMT‘s Facial Analysis Technologies adhere to very high data privacy and protection standards.

Features

  • Detection of faces and extraction of facial features and characteristics
  • Head pose estimation
  • Generation of face embeddings for face recognition
  • Classification of attributes such as gender or age
  • Tracking faces across consecutive video frames and selecting representative frames for further analysis
  • Face clustering: grouping by similar faces in unlabeled data
  • Face search: finding similar faces in unlabeled data based on a specific example
  • Face verification: Checking whether a face is in a database or not
  • Face recognition: recognizing a specific person in a labeled database

Range of Services

  • Consulting on the application and deployment of facial analysis technologies
  • Customized development and integration of facial analysis software solutions
  • Customized data model training and deployment
  • Evaluating machine learning methods and solutions
  • Benchmarking facial analysis technologies and best practice

Products and Use Cases

InsightPersona

InsightPersona helps to uncover media presence of politicians, celebrities and other individuals or groups on TV, in streams and video archives. Designed for broadcasters, media companies, and public authorities, our combined face and speaker analysis facilitates smarter program decisions by quantifying and visualizing data across thousands of hours of content. 

 

MaskCognizer

The AI-based MaskCognizer, developed by Fraunhofer IDMT, is a technology that detects and verifies in real time whether people are wearing their face masks correctly. It provides immediate and appropriate feedback for the protection of the general public and enables easier compliance with public health and safety regulations. We also offer other solutions that can support access management and the monitoring of customer and visitor flows in stores, restaurants or other facilities in a privacy-compliant manner.

IDCheck

The AI-based IDCheck facilitates secure, automated entry management to both buildings and events, greatly speeding up the time and effort required for identity verification and document control. It reliably and quickly verifies the identity of a person by visually matching their live camera image with the identity documents required for entry. 

 

Open Source Intelligence (OSINT)

Facial analysis technologies based on open source technologies help detect and recognize specific persons in image and video data. This approach is being used in the EU project, SPIRIT, which developed image-based inspection processes in the robotics domain. The OSINT approach aims to develop novel methods to resolve virtual identities back to real ones in compliance with data protection requirements, which can be used for law enforcement "intelligence analysis" purposes.

Automatic annotation of media content

The automatic detection and identification of faces contributes to the creation and enhancement of metadata for large image and video archives. For example, recognition software can automatically detect actors in videos or photos. This enables, for example, the fast retrieval of image and video sequences with known persons in large media and broadcast archives for more efficient content production or analysis purposes.

Biodiversiy monitoring

Gamekeepers use video traps and audio recording devices to investigate the behavior of animals threatened with extinction for biodiversity conservation management projects. In the SAISBECO project, a new type of biodiversity identification software was developed by Fraunhofer IDMT. This recognition software is capable of automatically searching through single images, video and audio recordings for sequences involving great apes. It is able to identify specific individuals, whether gorilla or chimpanzee. The software has the capability to sort the images accordingly to enable the identification and classification of animals across the ape species. The software may also be used for edutainment offers and interactive animal observation stations in zoos and wildlife parks.