models vorlaeufig fertig

project_report.md

@@ -109,12 +109,39 @@ The notebook comes with one class grid search for hyperparameter tuning as well
 Overall, our experiments have shown, that this approach is not sufficient, with the following results: 
 | Metric / Model | Isolation Forest | 
 |----------------|---------|
-| Best Balanced Accuracy |0.575|
-| Best AUC | 0.617 |
-In detail, the classificator tends to classify the majority of samples as low load. 
-### 4.4 OCSVM
-### 4.5 DeepSVDD
+| Best Balanced Accuracy |0.57|
+| Best AUC | 0.61|
 
+In detail, the classificator tends to classify the majority of samples as low load and is therefore not sufficient to be used for later deployment.
+
+
+### 4.4 One Class SVM with Autoencoder
+The training of an On Class SVM on the data from the dataset resulted in every sample was predicted as an anomaly. 
+In the next step, an autoencoder is pretrained to learn representation of the data. Afterwards, the encoder is used for preprocessing, which leads to OCSVM training on encoder output. 
+The training includes hyperparameter tuning through gridsearch cv. 
+Encoder output is visualized with print statements and plots that show the encoded data for both low and high load samples. 
+We see that the encoder struggles to represent the unseen high load samples differently. As a consequence, the One Class SVM also does not achieve sufficient performance. 
+| Metric / Model | One Class SVM | 
+|----------------|---------|
+| Best Balanced Accuracy |0.62|
+
+When the notebook is run completely, both the trained encoder and svm are saved for later use given the save paths are set correctly. 
+### 4.5 Deep SVDD
+Similar to the OCSVM training, an autoencoder is used to preprocess the data before the actual Deep SVDD training. Nevertheless, the usage is partialy different. The Dee SVDD uses a pretrained encoder to fine tune it by apllying a different loss function (which results from the theoretical concept behind Deep SVDD). This means that the encoder weights are still modified in the actual Deep SVDD training. 
+Also, this approach includes **hybrid fusion of modalities**. Instead of putting all features into the same input layer, the neural network is divided into two branches, that process action units and eye-tracking features separately.
+Then, after two Dense layers each, the branches are fusioned by concatenation. From there, another two Dense layers process the data. 
+The decoder is not exactly similar, as the split of the modalities happens are the very end. 
+To compute the total loss, loss from both modalities is combined by sum. Users are able to change loss weights. Training includes 2x2 phases, both autoencoder and later Deep SVDD are first trained with larger learning rate, then fine tuned with a smaller learning rate. 
+
+| Metric / Model | Deep SVDD | 
+|----------------|---------|
+| Best Balanced Accuracy |0.60|
+| Best AUC | 0.57|
+
+### 4.6 General information on unsupervised approaches
+As described above, the approachs didn't meet the requirements in terms of prediction performance. For all models, both MinMax-Scaling as well as Standard-Scaling was done. Also, both subjectwise and globally. Unfortunately, the differences were not that large, which may explain why preprocessing wasn't mentioned above. 
+
+Future research should always keep in mind while subject-wise scaling might be better for training, it makes deployment on new subjects more difficult. Our solution, as implemented in `model_training/tools/scaler.py` calculates a fallback scaler (using mean of all subjects scaling parameters). 
 
 ## 5) Real-Time Prediction and Messaging