Developing a Hybrid driver Drowsiness Detection System utilizing Deep Learning techniques

다국어 초록 (Multilingual Abstract)

Driver drowsiness is a leading cause of road accidents globally, necessitating the development of real-time monitoring systems to enhance road safety. Current drowsiness detection methods often rely on analyzing physiological indicators such as eye movements and heart rate variability or observing driver behavior like lane drifting. While these methods show promise, they face significant challenges in real-world environments, particularly when drivers are subjected to poor lighting, wear facial accessories like glasses, or have varying head positions. These factors reduce the accuracy and reliability of traditional systems, prompting the need for advanced machine learning approaches to adapt to these challenges.
This thesis introduces a novel driver drowsiness detection system that integrates deep learning models, including Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). By combining these models, the system improves detection under varying environmental conditions and driver behaviors. A key innovation is the use of YuNet for face detection, which handles head pose disorientations and functions effectively in scenarios involving occlusion, lighting variations, and facial accessories. YuNet offers a lightweight, efficient face detection solution suitable for both GPU and CPU deployment, addressing the computational limitations faced by other systems.
At the core of the system is a hybrid model that integrates InceptionV3, DenseNet121, and Vision Transformers (ViTs), enabling it to capture both localized and global features from driver faces. This hybrid approach significantly improves the system's accuracy in classifying driver states as either "Active" or "Drowsy." Our modified ViT(40x40) model, with larger patch sizes and enhanced image resolution, further demonstrated superior performance in both controlled environments and real-world scenarios.
The hybrid model excelled across all metrics, achieving near-perfect accuracy in controlled environments and maintaining robustness in challenging conditions such as low lighting and facial occlusion. On the custom dataset, the hybrid model achieved a test accuracy of 98.11%, demonstrating its generalizability to real-world scenarios. One notable strength of the system is its efficiency; YuNet’s optimized design enables real-time detection on both GPU and CPU devices, making the system viable for use in vehicles with limited computing power.

목차 (Table of Contents)

• I. Introduction 1
• 1. Background of the Study 1
• 2. Existing 6
• 3. Problem Statement. 7
• 4. Objectives & Goals. 8
• 5. Research Question. 9
• 6. Impact & Significance of the Study 10
• II. Literature review 12
• 1. Subjective measures 13
• 2. Physiological measures 18
• 3. Vehicle-based measures 28
• 4. Behavioral measures 33
• 1.1. Facial Landmarks Approach. 34
• 1.2. Deep learning approach. 41
• III. Methodology 65
• 1. Proposal System Overview and Methodology Introduction. 65
• 2. System Configuration 67
• 3. Data Collection and Preprocessing 67
• 3.1 Types of data and collection. 67
• 3.2 Data Balancing and Distribution. 71
• 3.3 Face Extraction for Training the Classification Model 72
• 3.4 Data Augmentation Process 74
• 4. Face Detection Models 76
• 4.1 Face Detection Models Overview 76
• 4.2 Comaprison of the Existing models 85
• 4.3 Selection of YuNet for Face Detection: Justification and Rationale 90
• 5. Drowsiness Detection Model 91
• 5.1 CNN models. InceptionV3 and DenseNet121 92
• 5.2 Vision Transformers 95
• 5.3 Multi-Scale Feature Extraction Using a Hybrid Model Based on CNN and ViT 102
• 6. Model Evaluation 107
• IV. Results and Discussion 110
• 1. Drowsiness Detection models’ results 110
• 1.1 CNN models 110
• 1.2. Vision Transformers 118
• 1.3 Multiscale Feature Extraction Using a Hybrid Model Based on InceptionV3, DenseNet121, and ViT 124
• 2. Discussion 131
• V. Technology Comparison and Proposal System’s performance overview 136
• 1. Performance Evaluation of the Proposal Hybrid Drowsiness Detection System in Varying Conditions 136
• 2. Technology comparison 140
• VI. Conclusion and Recommendations 143
• VII. Action Plan 146 VIII. References 172