AI-Based Driver State Detection

In the Context of Handover Situations of (Partially) Autonomous Driving

Project Justification

In the coming decades, automated vehicles are expected to become increasingly integrated into everyday traffic. However, human drivers will continue to play a crucial role in monitoring the driving environment and taking over control when necessary.

A key element in this context is the ability to execute a safe and timely handover. Driver readiness refers to the driver’s capacity to resume control of the vehicle in a given situation and to respond appropriately. This readiness is therefore a fundamental prerequisite for safe and effective interaction between driver and automation, and essential for ensuring overall system reliability in partially and conditionally automated driving scenarios.

Possible experimental setup for measuring driver’s facial expressions, eyelid movements, posture, and seat position

Our Approach

The DrAIve project – funded by the Federal Ministry of Education and Research (BMBF) – aims to make driver readiness measurable in real time. To this end, an AI-based driver assistance system is being developed that automatically classifies driver characteristics and behaviors.

Audio and video data – including facial expressions, eyelid movements, heart rate, voice, posture, and seat position – are used to generate predictions about the driver’s readiness to take over control.

By continuously analyzing both the driver and the vehicle interior, the system can detect critical driving conditions within a very short time frame. DrAIve thus contributes to reducing high-risk situations in road traffic by enabling timely and targeted safety interventions.

IXP test track in Datteln

Work Packages, Insights and Outcomes

IXP leads the requirements analysis by identifying user needs as well as technical and methodological prerequisites for future data classification and measurement systems. In this context, IXP is responsible for defining the foundational conditions necessary for reliable and scalable driver state detection.

Furthermore, IXP oversees corpus engineering through both laboratory and field studies – ensuring high data quality for subsequent model development and evaluation.

In the development phase, IXP supports project partners in building the audio-, video-, and cabin activity–based driver state detection module, with a particular focus on training deep learning models and evaluating initial prototypes.

Finally, IXP conducts functionality testing, usability studies, and iterative refinements to ensure the reliability, effectiveness, and user-centered quality of the system – thereby contributing substantially to the overall success of DrAIve.

Overview of the framework, hypotheses, and proposed approach of the DrAIve Project

Related Projects