Camera-Based Monitoring of Safety-Critical Driver Conditions

Feasibility Study for the Development of a Deep-Learning-Based Driver Assistance System

Project Justification

The goal of this project was to investigate the state of the art in driver’s state recognition. Further, it aimed to identify prerequisites for the acceptance and successful implementation of assistance systems. Safety and comfort are important features in today’s vehicles. Increasing requirements for safety encourage manufacturers and suppliers to further develop innovative driver assistance systems. Moreover, the assessment of the driver’s state is an important component in automated driving. Comfort- and safety-enhancing add-on sensor systems also have an increasing impact on the image of manufacturers and are proving to be an important differentiator between brands.

Schematic structure of the audio-video-based VAS for monitoring critical driver states (e.g. stress, fatigue, distraction, aggression). The sensor data are obtained from (a) voice, (b) heart rate, (c) 3D head and eye movement and (d) facial expression, then processed by a deep learning model and finally derived as an “assessment score” regarding the driver’s state.

Our Approach

DeepSafeCam aimed to create a full picture on the acceptance and feasibility of a camera-based driver’s state assistance system. Corresponding aspects were assessed in regard to a multifunctional prototype with one audio- and three video-based modules. This approach allows simultaneous, more accurate, and robust monitoring of critical stress, fatigue, distraction, and aggression. It is further accompanied by novel deep learning algorithms that enable real-time data analyses and processing as well as final outputs of driver states. The prototype was evaluated through a market analysis, a technological risk assessment, and a legislation risk analysis. User acceptance was evaluated via focus groups and psychometrical measures.

Prototype for identifying driver states such as fatigue, inattention, and aggression.

Work Packages, Insights and Outcomes

Ixp was responsible for conducting a market analysis to investigate the customer’s acceptance of a camera-based driver assistance system. Further, ixp administered a technological risk assessment focusing on systems which are non-invasive, accurate, robust, and maintenance free . Similarly, a legislation risk analysis was conducted assessing ethical, legal, and social implications in regard to data privacy. In sum, this project delivered new insights into the user, legal, and technical requirements of driver monitoring systems.