diff --git a/project_report.md b/project_report.md index 5da6ccb..e8a7976 100644 --- a/project_report.md +++ b/project_report.md @@ -7,18 +7,20 @@ The system combines: - Facial Action Units (AUs) - Eye-tracking features (fixations, saccades, blinks, pupil behavior) -It covers: -- Data ingestion and conversion -- Sliding-window feature generation -- Exploratory analysis +Apart from this, several machine learning model architectures are presented and evaluated. + +Content: +- Dataset generation +- Exploratory data analysis - Model training experiments -- Real-time inference from SQLite -- MQTT publishing -- Optional Linux `systemd` scheduling +- Real-time inference with SQlite, systemd and MQTT +- Repository file inventory +- Additional nformation -## 2) End-to-End Workflow -### 2.1 Data Ingestion and Conversion +## 2) Dataset generation + +### 2.1 Data Access, Filtering, and Data Conversion Main scripts: - `dataset_creation/create_parquet_files_from_owncloud.py` @@ -59,7 +61,18 @@ Runtime behavior: Operational note: - `DB_PATH` and other paths are currently code-configured and must be adapted per deployment. -### 2.4 Model Training +## 3) EDA +TO DO + +- `EDA/EDA.ipynb` - main EDA notebook +- `EDA/distribution_plots.ipynb` - distribution visualization +- `EDA/histogramms.ipynb` - histogram analysis +- `EDA/researchOnSubjectPerformance.ipynb` - subject-level analysis +- `EDA/owncloud_file_access.ipynb` - ownCloud exploration/access notebook +- `EDA/calculate_replacement_values.ipynb` - fallback/median computation notebook +- `EDA/login.yaml` - local auth/config artifact for EDA workflows + +## 4) Model Training Location: - `model_training/` (primarily notebook-driven) @@ -77,7 +90,7 @@ Supporting utilities: - `model_training/tools/mad_outlier_removal.py` - `model_training/tools/evaluation_tools.py` -### 2.5 Real-Time Prediction and Messaging +## 5) Real-Time Prediction and Messaging Main script: - `predict_pipeline/predict_sample.py` @@ -99,7 +112,7 @@ Expected payload form: } ``` -### 2.6 Scheduled Prediction (Linux) +### 5.1 Scheduled Prediction (Linux) Files: - `predict_pipeline/predict.service` @@ -110,7 +123,7 @@ Role: - Run inference repeatedly without manual execution - Timer/service configuration can be customized per target machine -## 3) Runtime Configuration +## 5.2 Runtime Configuration Primary config file: - `predict_pipeline/config.yaml` @@ -127,7 +140,7 @@ Important: - The repository currently uses environment-specific absolute paths in some scripts/configs. - Paths should be normalized before deployment to a new machine. -## 4) Data and Feature Expectations +## 5.3) Data and Feature Expectations Prediction expects SQLite rows containing: - `_Id` @@ -141,8 +154,13 @@ Common feature groups: - Blink count/duration statistics - Pupil mean and IPA -## 5) Installation and Dependencies +## 6) Installation and Dependencies +Due to unsolvable dependency conflicts, several environemnts need to be used in the same time. +### 6.1 Environemnt for camera handling +TO DO + +### 6.2 Environment for predictions Install base requirements: ```bash pip install -r requirements.txt @@ -154,16 +172,16 @@ Typical key packages in this project: - `paho-mqtt` - optional data access stack (`pyocclient`, `h5py`, `tables`) -## 6) Repository File Inventory +## 7) Repository File Inventory -### 6.1 Root +### Root - `.gitignore` - Git ignore rules - `readme.md` - minimal quickstart documentation - `project_report.md` - full technical documentation (this file) - `requirements.txt` - Python dependencies -### 6.2 Dataset Creation +### Dataset Creation - `dataset_creation/parquet_file_creation.py` - local source to parquet conversion - `dataset_creation/create_parquet_files_from_owncloud.py` - ownCloud download + parquet conversion @@ -182,7 +200,7 @@ Typical key packages in this project: - `dataset_creation/camera_handling/camera_stream.py` - baseline camera streaming script - `dataset_creation/camera_handling/db_test.py` - DB test utility -### 6.3 EDA +### EDA - `EDA/EDA.ipynb` - main EDA notebook - `EDA/distribution_plots.ipynb` - distribution visualization @@ -192,7 +210,7 @@ Typical key packages in this project: - `EDA/calculate_replacement_values.ipynb` - fallback/median computation notebook - `EDA/login.yaml` - local auth/config artifact for EDA workflows -### 6.4 Model Training +### Model Training #### CNN - `model_training/CNN/CNN_simple.ipynb` @@ -233,7 +251,7 @@ Typical key packages in this project: - `model_training/tools/mad_outlier_removal.py` - `model_training/tools/evaluation_tools.py` -### 6.5 Prediction Pipeline +### Prediction Pipeline - `predict_pipeline/predict_sample.py` - runtime prediction + MQTT publish - `predict_pipeline/config.yaml` - runtime database/model/scaler/mqtt config @@ -242,12 +260,12 @@ Typical key packages in this project: - `predict_pipeline/predict.timer` - systemd timer unit - `predict_pipeline/predict_service_timer_documentation.md` - Linux service/timer guide -### 6.6 Generic Tools +### Generic Tools -- `tools/db_helpers.py` - common SQLite utilities used by prediction path +- `tools/db_helpers.py` - common SQLite utilities used to get newest sample for prediction -## 7) Known Technical Notes +## 8) Additional Information -- Several paths are hardcoded for a specific runtime environment and should be parameterized for portability. +- Several paths are hardcoded on purpose to ensure compability with the jetsonboard at the OHM-UX driving simulator. - Camera and AU processing are resource-intensive; version pinning and hardware validation are recommended.