Multimodal Driver State Analysis

Short overview: this repository contains the data, feature, training, and inference pipeline for multimodal driver-state analysis using facial AUs and eye-tracking signals.

For full documentation, see project_report.md.

Quickstart

1) Setup

Activate the conda-repository "".

    conda activate

Make sure, another repository that fulfills requirement.txt is available, matching with predict_pipeline/predict.service - See predict_pipeline/predict_service_timer_documentation.md

2) Camera AU + Eye Pipeline (camera_stream_AU_and_ET_new.py)

1. Open dataset_creation/camera_handling/camera_stream_AU_and_ET_new.py and adjust:

• DB_PATH
• CAMERA_INDEX
• OUTPUT_DIR (optional)

2. Start camera capture and feature extraction:

python dataset_creation/camera_handling/camera_stream_AU_and_ET_new.py

3. Stop with q in the camera window.

3) Predict Pipeline (predict_pipeline/predict_sample.py)

1. Edit predict_pipeline/config.yaml and set:

• database.path, database.table, database.key
• model.path
• scaler.path (if use_scaling: true)
• MQTT settings under mqtt

2. Run one prediction cycle:

python predict_pipeline/predict_sample.py

3. Use predict_service_timer_documentation.md to see how to use the service and timer for automation.