init

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/Doc_BoxAlgorithm_ClassifierProcessor.dox-part

@@ -0,0 +1,90 @@
+/**
+ * \page BoxAlgorithm_ClassifierProcessor Classifier processor
+__________________________________________________________________
+
+Detailed description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Description|
+The <em>Classifier Processor</em> box is a generic box for classifying data (feature vectors). 
+It works in conjunction with the \ref Doc_BoxAlgorithm_ClassifierTrainer box.
+This box' role is to expose a generic interface to the rest of the BCI pipeline. The 
+vectors to classify are forwarded to an algorithm or a structure of algorithms depending on what is
+described in the loaded configuration file. The behavior is simple: at initialization phase, the classification 
+structure is initialized and its configuration is loaded from the configuration file. Then each time this box 
+receives a new feature vector, it is forwarded to the classification algorithm that classifies it. The box gets the algorithm
+status and the actual class value and translates this information to its output. The predicted class is sent out in 
+the form of a stimulation and the algorithm status is sent in the form a streamed matrix. The stimulation can be generically
+interpreted by the rest of the pipeline but it is important to understand that each classification algorithm is
+free to report whatever it wants in its "status matrix". Consequently, the use of this output stream will be
+dependent on the chosen classification algorithm. For example, the LDA classifier sends the hyperplane distance
+value as its status.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Description|
+__________________________________________________________________
+
+Inputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Inputs|
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Inputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Input1|
+This input should be connected to the feature vector stream to classify. Each time a new feature vector arrives,
+a classification process will be triggered. Consequently, a classification stimulation will be sent on the
+first output of this box.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Input1|
+__________________________________________________________________
+
+Outputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Outputs|
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Outputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Output1|
+This output will contain the classification stimulations. Each time a new feature vector arrives to this box,
+a new classification process is triggered, resulting in the generation of the corresponding class stimulation.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Output1|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Output2|
+This output reflects the classification algorithm status in the form of a matrix of value.  This output will contain one or several distances
+to an hyperplane if the classifier provide it. If not, the matrix will have 0 dimension. The format of this output directly depend on 
+the classification algorithm and of the strategy used by the processor box.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Output2|
+ * 
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Output3|
+This output reflects the classification algorithm status in the form of a matrix of value.  This output will contains one or several probabilities
+for a data to be on a class if the classifier provide it. If not, the matrix will have 0 dimension. The format of this output directly depend on 
+the classification algorithm and of the strategy used by the processor box.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Output2|
+__________________________________________________________________
+
+Settings description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Settings|
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Settings|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Setting1|
+This setting points to the configuration file of the box generated by the 
+\ref Doc_BoxAlgorithm_ClassifierTrainer box. Its syntax depends on the selected algorithm.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Setting1|
+
+__________________________________________________________________
+
+Examples description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Examples|
+This box is used in BCI pipelines in order to classify cerebral activity states. For a detailed scenario using this
+box and its associated \ref Doc_BoxAlgorithm_ClassifierTrainer, please see the <b>motor imagary</b>
+BCI scenario in the sample scenarios.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Examples|
+__________________________________________________________________
+
+Miscellaneous description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierProcessor_Miscellaneous|
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierProcessor_Miscellaneous|
+ */

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/Doc_BoxAlgorithm_ClassifierTrainer.dox-part

@@ -0,0 +1,285 @@
+/**
+ * \page BoxAlgorithm_ClassifierTrainer Classifier trainer
+__________________________________________________________________
+
+Detailed description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Description|
+The <em>Classifier Trainer</em> box is a generic box for training models to classify input data. 
+It works in conjunction with the \ref Doc_BoxAlgorithm_ClassifierProcessor box.
+This box' role is to expose a generic interface to the rest of the BCI pipelines. The box 
+will generate an internal structure according to the multiclass strategy and the learning
+algorithm selected.
+
+The behavior is simple, the box collects a number of feature vectors. Those feature vectors
+are labelled depending on the input they arrive on. When a specific stimulation arrives, a training
+process is triggered. This process can take some time so this box should be used offline. Depending on the
+settings you enter, you will be able to perform a k-fold test to estimate the accuracy of the learned
+classifier. When this training stimulation is received, the box generates a configuration file that will 
+be usable online by the \ref Doc_BoxAlgorithm_ClassifierProcessor box. 
+Finally, the box outputs a particular stimulation (OVTK_StimulationId_TrainCompleted) 
+on its output, that can be used to trigger further treatments in the scenario.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Description|
+__________________________________________________________________
+
+Inputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Inputs|
+This box can have a variable number of inputs. If you need more than two classes, feel free to add more
+inputs and to use a proper strategy/classifier combination to handle more than two classes.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Inputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Input1|
+The first input receives a stimulation stream. Only one stimulation of this stream is important, the one
+that triggers the training process. When this stimulation is received, all the feature vectors are labelled
+and sent to the classification algorithm. The training is triggered and executed. Then the classification
+algorithm generates a configuration file that will be used online by the \ref Doc_BoxAlgorithm_ClassifierProcessor box.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Input1|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Input2|
+This input receives the feature vector for the first class.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Input2|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Input3|
+This input receives the feature vector for the second class.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Input3|
+__________________________________________________________________
+
+Outputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Outputs|
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Outputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Output1|
+The stimulation OVTK_StimulationId_TrainCompleted is raised on this output when the classifier trainer has finished its job.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Output1|
+
+__________________________________________________________________
+
+Settings description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Settings|
+The number of settings of this box can vary depending on the classification algorithm you choose. Such algorithm
+could have specific input OpenViBE::Kernel::IParameter objects (see \ref OpenViBE::Kernel::IAlgorithmProxy for details). If
+the type of those parameters is simple enough to be handled in the GUI, then additional settings will be added to this box.
+<b>After switching a strategy or a classifier, you will have to close and re-open the settings configuration dialog to see the parameters of the new classifier.</b> Supported parameter types are : Integers, Floats, Enumerations, Booleans. The documentation for those
+parameters can not be done in this page because it is impossible to know at this time what classifier thus what hyper
+parameters you will have available. This will depend on the classification algorihtms that are be implemented in OpenViBE.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Settings|
+ * 
+ *  * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting1|
+The stimulation that triggers the training process and save the learned classifier to disk.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting1|
+ * 
+ *  * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting2|
+This setting points to the configuration file where to save the result of the training for later online use. This
+configuration file is used by the \ref Doc_BoxAlgorithm_ClassifierProcessor box. Its syntax
+depends on the selected algorithm.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting2|
+
+  * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting3|
+This setting is the strategy to use. You can choose any registered \c OVTK_TypeId_ClassificationStrategy
+strategy you want.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting3|
+ * 
+ *  * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting4|
+This is the stimulation to send when the classifier algorithm detects a class-1 feature vector	
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting4|
+ * 
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting5|
+This is the stimulation to send when the classifier algorithm detects a class-2 feature vector	
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting5|
+ 
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting6|
+This setting is the classifier to use. You can choose any registered \c OVTK_TypeId_ClassifierAlgorithm
+algorithm you want.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting6|
+
+
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting10|
+If you want to perform a k-fold test, you should enter something else than 0 or 1 here. A k-fold test generally gives
+a better estimate of the classifiers accuracy than naive testing with the training data. The classifier may overfit
+the training data, and get a good accuracy with the observed data, but not be able to generalize to unseen data. 
+In cross-validation, the idea is to divide the set of  feature vectors in a number of partitions. The classification algorithm 
+is trained on some of the partitions and its accuracy is tested on the others. However, the classifier produced by the box is 
+the classifier trained with the whole data. The cross-validation is only an error estimation tool, it does not affect 
+the resulting model. See the miscellaneous section for details on how the k-fold test is done in this box, and possible 
+caveats about the cross-validation procedure.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting10|
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Setting11| 
+If the number of class labels is unbalanced, the classifiers tend to be biased towards the majority labels.
+This option can be used to resample the dataset to feature all classes equally. 
+
+The algorithm first looks how many examples there are in the majority class. Lets say this is n. Then, if class k has m examples, 
+it will random sample n-m examples with replacement from class k, appending them to the dataset. This will be done for each class. 
+In the end, each class will have n examples and all except the majority class will have some duplicate training vectors. 
+This can be seen as a technique to weight the importance of examples for such classifiers that do not support setting example weights 
+or class weight prior, and can in general be attempted with arbitrary learning algorithms.
+
+Enabling this option may make sense if the box is used for incremental learning, where all classes may not be equally represented 
+in the training data obtained so far, even if the design itself is balanced. Note that enabling this will make the cross-validation 
+results optimistic. In most conditions, the feature should be disabled.
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Setting11|
+
+__________________________________________________________________
+
+Examples description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Examples|
+This box is used in BCI pipelines in order to classify cerebral activity states. For a detailed scenario using this
+box and its associated \ref Doc_BoxAlgorithm_ClassifierProcessor, please see the <b>motor imagary</b>
+BCI scenario in the sample scenarios. An even more simple tutorial with artificial data 
+is available in the <b>box-tutorials/</b> folder.
+
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Examples|
+__________________________________________________________________
+
+Miscellaneous description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_ClassifierTrainer_Miscellaneous|
+
+The box supports various multiclass strategies and classifiers as plugins.
+ 
+\par Available strategy:
+Strategy refers to how feature vectors are routed to one or more classifiers, which possibly can handle only 2 classes themselves.
+
+\par Native
+Use the classifier training algorithm without a pairwise strategy. All the data is passed to a single classifier trainer.
+
+\par One Vs All
+Use a pairwise strategy which consists of training each class against all the others, creating n classifiers for n classes.
+
+\par One vs One
+Use a airwise strategy which trains one classifier for each pair of classes. Then we use a decision startegy to extract the most likely class. There are three differents decision strategy:
+\li Voting: method based on a simple majority voting process
+\li HT: method described in: Hastie, Trevor; Tibshirani, Robert. Classification by pairwise coupling. The Annals of Statistics 26 (1998), no. 2, 451--471
+\li PKPD: method describe in: Price, S. Knerr, L. Personnaz, and G. Dreyfus. Pairwise neural network classifiers with probabilistic outputs. In G. Tesauro, D. Touretzky, and T. Leen (eds.)
+Advances in Neural Information Processing Systems 7 (NIPS-94), pp. 1109-1116. MIT Press, 1995.
+You cannot use every algorithm with every decision strategy, but the interface will restain the choice according to your selection.
+ 
+ 
+ 
+\par Available classifiers:
+
+\par Support Vector Machine (SVM)
+A well-known classifier supporting non-linear classification via kernels. The implementation is based on LIBSVM 2.91, which is included in the OpenViBE source tree. The parameters exposed in the GUI correspond to LIBSVM parameters. For more information on LIBSVM, see <a href="http://www.csie.ntu.edu.tw/~cjlin/libsvm/">here</a>.
+\par 
+This algorithm provides only probabilities.
+
+\par Linear Discriminant Analysis (LDA)
+ A simple and fast linear classifier. For description, see any major textbook on Machine Learning or Statistics (e.g. Duda, Hart & Stork, or Hastie, Tibshirani & Friedman). This algorithm can be used with a regularized covariance matrix
+ according to a method proposed by Ledoit & Wolf: "A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices", 2004.
+ The Linear Discriminant Analysis has the following options.
+ \par
+ \li Use shrinkage: Use a classic or a regularized covariance matrix.
+ \li Shrinkage: A value s between [0,1] sets a linear weight between dataCov and priorCov. I.e. cov=(1-s)*dataCov+s*priorCov.
+ Value <0 is used to auto-estimate the shrinking coefficient (default). If var(x) is a vector of empirical variances of all data dimensions, priorCov is a 
+ diagonal matrix with a single value mean(var(x)) pasted on its diagonal. Used only if use shrinkage is checked.
+\li Force diagonal cov (DDA): This sets the nondiagonal entries of the covariance matrices to zero. Used only if Use shrinkage is checked.
+\par
+Note that setting shrinkage to 0 should get you the regular LDA behavior. If you additionally force the covariance to be diagonal, you should get a model resembling the Naive Bayes classifier.
+\par 
+This algorithm provides both hyperplane distance and probabilities.
+
+\par Multilayer Perceptron (MLP)
+A classifier algorithm which relies on an artificial neural network (<a href="https://hal.inria.fr/inria-00099922/en">Laurent Bougrain. Practical introduction to artificial neural networks. IFAC symposium on automation in Mining, Mineral and Metal Processing - 
+MMM'04, Sep 2004, Nancy, France, 6 p, 2004.</a>). In OpenViBE, the MLP is a 2-layer neural network. The hyperbolic tangent is the activation function of the
+neurons inside the hidden layer. The network is trained using the backpropagation of the gradient. During the training, 80% of the training set is used to compute the gradient, 
+and 20% is used to validate the new model. The different weights and biases are updated only once per iteration (just before the validation). A coefficient alpha (learning coefficient) is used to moderate the importance of 
+the modification of weights and biases to avoid oscillations. The learning stops when the difference of the error per element (computed during validation) of two consecutive iterations is under the value epsilon given as a parameter.
+\par
+\li Number of neurons in hidden layer: number of neurons that will be used in the hidden layer.
+\li Learning stop condition : the epsilon value used to stop the learning
+\li Learning coefficient: a coefficient which influence the speed of learning. The smaller the coefficient is, the longer the learning will take, the more chance you will have to get a good solution.
+\par
+Note that feature vectors are normalized between -1 and 1 (using the min/max of the training set) to avoid saturation of the hyperbolic tangent.
+\par
+This algorithm provides both hyperplane distance (identity of output layer) and probabilites (softmax function on output layer).
+ 
+\par Cross Validation 
+ 
+In this section, we will detail how the k-fold test is implemented in this box. For the k-fold test to be performed, you
+have to choose more than 1 partition in the related settings. Suppose you chose \c n partitions. Then when trigger stimulation
+is received, the feature vector set is splitted in \c n consecutive segments. The classification algorithm is trained on
+\c n-1 of those segments and tested on the last one. This is performed for each segment. 
+
+For example, suppose you have 5 partitions of feature vectors (\c FVs)
+\verbatim
++------+ +------+ +------+ +------+ +------+
+| FVs1 | | FVs2 | | FVs3 | | FVs4 | | FVs5 |
++------+ +------+ +------+ +------+ +------+
+\endverbatim
+For the first training, a feature vector set is built form the \c FVs2, \c FVs3, \c FVs4, \c FVs5. The classifier algorithm
+is trained on this feature vector set. Then the classifier is tested on the \c FVs1 :
+\verbatim
++------+ +---------------------------------+
+| FVs1 | |  Training Feature Vector Set 1  |
++------+ +---------------------------------+
+\endverbatim
+Then, a feature vector set is built form the \c FVs1, \c FVs3, \c FVs4, \c FVs5. The classifier algorithm
+is trained on this feature vector set. Then the classifier is tested on the \c FVs2 :
+\verbatim
++-------+ +------+ +------------------------+
+| Train | | FVs2 | | ing Feat. Vector Set 2 |
++-------+ +------+ +------------------------+
+\endverbatim
+The same process if performed on all the partitions :
+\verbatim
++---------------+ +------+ +---------------+
+|Training Featur| | FVs3 | |e Vector Set 3 |
++---------------+ +------+ +---------------+
++------------------------+ +------+ +------+
+|Training Feature Vector | | FVs4 | |Set 4 |
++------------------------+ +------+ +------+
++---------------------------------+ +------+
+|  Training Feature Vector Set 5  | | FVs5 |
++---------------------------------+ +------+
+\endverbatim
+
+Important things to consider :
+- The more partitions you have, the more feature vectors you have in your training sets... and the less examples 
+you'll have to test on. This means that the result of the test will probably be less reliable. 
+
+In conclusion, be careful when choosing this k-fold test setting. Typical value range from 4 partitions (train on 75% of the feature vectors and
+test on 25% - 4 times) to 10 partitions (train on 90% of the feature vectors and test on 10% - 10 times).
+
+Note that the cross-validation performed by the classifier trainer box in OpenViBE may be optimistic.
+The cross-validation computation is working as it should, but it cannot take into account what happens outside
+the classifier trainer box. In OpenViBE scenarios, there may be e.g. time overlap from epoching, feature
+vectors drawn from the same epoch ending up in the same cross-validation partition, and (supervised) 
+preprocessing such as CSP or xDAWN potentially overfitting the data before its given to the classifier trainer. 
+Such situations are not compatible with the theoretical assumption that the feature vectors are 
+independent and identically distributed (the typical iid assumption in machine learning) across
+train and test. To do cross-validation controlling for such issues, we have provided
+a more advanced cross-validation tutorial as part of the OpenViBE web documentation.
+
+\par Confusion Matrices
+
+At the end of the training, the box will print one or two confusion matrices, depending if cross-validation
+was used: one matrix for the cross-validation, the other for the training data. Each matrix will contain true 
+class as rows, and predicted class as columns. The diagonal describes the percentage of correct predictions per class. 
+Although the matrix can be optimistic (see above section about the cross-validation), it may give useful 
+diagnostic information. For example, if the accuracy is very skewed towards one class, this may indicate
+a problem if the design is supposed to be balanced. The problem may originate e.g. from the original data 
+source, the signal processing chains for the different classes, or the classifier learning algorithm. These need
+then to be investigated. Also, if very low accuracies are observed in these matrices, it may give reason 
+to suspect that prediction accuracies on fresh data might be likewise lacking -- or worse.
+
+\par Incremental Learning
+
+The box can also be used for simple incremental (online) learning. To achieve this, simply send the box the training
+stimulation and it will train a classifier with all the data it has received so far. You can give it more
+feature vectors later, and trigger the learning again by sending another stimulation. Likewise, the corresponding 
+classifier processor box can be made to load new classifiers during playback. With classifiers like LDA, 
+this practice is usually feasible when the data is reasonably sized (as in basic motor imagery).
+
+ * |OVP_DocEnd_BoxAlgorithm_ClassifierTrainer_Miscellaneous|
+ */

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/Doc_BoxAlgorithm_VotingClassifier.dox-part

@@ -0,0 +1,112 @@
+/**
+ * \page BoxAlgorithm_VotingClassifier Voting Classifier
+__________________________________________________________________
+
+Detailed description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Description|
+The purpose of this simple classifier is to choose between multiple two class classifiers which
+one mostly fits a condition. For example imagine \c n states. Each of those states can be either active
+or inactive. Additionally, imagine you want only one active state at a time. Then you can have \c n
+two-class classifiers telling for each state if it is active or not, and a following voting classifier
+that chooses which of those states is the <em>most active</em>.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Description|
+__________________________________________________________________
+
+Inputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Inputs|
+You can add as many inputs as you need depending on the number of preceeding states.
+
+The inputs of this classifier can be changed to either streamed matrix of stimulations.
+
+In the case you choose stimulations, each active stimulation gives a point
+to the preceeding state. Both an inactive and a reject stimulation gives no point.
+Any other stimulation is ignored.
+After a number of repetitions, the state with the best score is chosen.
+
+In the case you choose streamed matrix, the matrix must have only one
+element. This element is used as a score coefficient (in place of the simple 1-0 of the previous
+case).  After a number of repetitions, the state with the best score is chosen.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Inputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Input1|
+Input stream for the first state.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Input1|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Input2|
+Input stream for the second state.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Input2|
+__________________________________________________________________
+
+Outputs description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Outputs|
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Outputs|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Output1|
+This output sends a new stimulation as soon as the classifier received the correct number of votes
+from the preceeding states. The output stimulation is based on the 5th setting of the box. First
+state being selected would send exactly this stimulation. Second state would send this
+stimulation + 1 etc.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Output1|
+__________________________________________________________________
+
+Settings description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Settings|
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Settings|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting1|
+This setting tells the box how many votes it needs before choosing. If the box uses stimulations,
+it must receive either the target or non target stimulation to consider a state has been voted.
+Additionaly, the box waits each state to be voted the correct number of times to take a decision.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting1|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting2|
+If the box uses stimulations, this settings tells what stimulation reflects that the state is active.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting2|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting3|
+If the box uses stimulations, this settings tells what stimulation reflects that the state is inactive.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting3|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting4|
+In case no choice can be made (for example, there are more than one state with the highest score), the
+voting classifier can choose to reject the vote and send a specific stimulation for this. This is more
+likely to happen when the box works on stimulation better than streamed matrix. You can force this box
+to choose using the 6th setting.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting4|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting5|
+This stimulation is used as a basis for the stimulations to send when a state is selected. First
+state being selected would send exactly this stimulation. Second state would send this
+stimulation + 1 etc.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting5|
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Setting6|
+This setting can force the box to choose a state even if more than one state have the highest score.
+In such case, there won't be any rejection. The way the box decides between the ex-aequo candidates
+is undefined.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Setting6|
+__________________________________________________________________
+
+Examples description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Examples|
+This box is used in the <em>P300 speller</em> and the <em>P300 magic card</em> BCIs. Please see those
+scenarios in the sample <em>openvibe-scenarios</em>.
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Examples|
+__________________________________________________________________
+
+Miscellaneous description
+__________________________________________________________________
+
+ * |OVP_DocBegin_BoxAlgorithm_VotingClassifier_Miscellaneous|
+ * |OVP_DocEnd_BoxAlgorithm_VotingClassifier_Miscellaneous|
+ */

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/boxes/Doc_BoxAlgorithm_ClassifierProcessor.rst

@@ -0,0 +1,101 @@
+.. _Doc_BoxAlgorithm_ClassifierProcessor:
+
+Classifier processor
+====================
+
+.. container:: attribution
+
+   :Author:
+      Yann Renard, Guillaume Serriere
+   :Company:
+      INRIA/IRISA
+
+.. image:: images/Doc_BoxAlgorithm_ClassifierProcessor.png
+
+Classifies incoming feature vectors using a previously learned classifier.
+
+The *Classifier Processor* box is a generic box for classifying data (feature vectors). 
+It works in conjunction with the :ref:`Doc_BoxAlgorithm_ClassifierTrainer` box.
+This box' role is to expose a generic interface to the rest of the BCI pipeline. The 
+vectors to classify are forwarded to an algorithm or a structure of algorithms depending on what is
+described in the loaded configuration file. The behavior is simple: at initialization phase, the classification 
+structure is initialized and its configuration is loaded from the configuration file. Then each time this box 
+receives a new feature vector, it is forwarded to the classification algorithm that classifies it. The box gets the algorithm
+status and the actual class value and translates this information to its output. The predicted class is sent out in 
+the form of a stimulation and the algorithm status is sent in the form a streamed matrix. The stimulation can be generically
+interpreted by the rest of the pipeline but it is important to understand that each classification algorithm is
+free to report whatever it wants in its "status matrix". Consequently, the use of this output stream will be
+dependent on the chosen classification algorithm. For example, the LDA classifier sends the hyperplane distance
+value as its status.
+
+Inputs
+------
+
+.. csv-table::
+   :header: "Input Name", "Stream Type"
+
+   "Features", "Feature vector"
+   "Commands", "Stimulations"
+
+Features
+~~~~~~~~
+
+This input should be connected to the feature vector stream to classify. Each time a new feature vector arrives,
+a classification process will be triggered. Consequently, a classification stimulation will be sent on the
+first output of this box.
+
+Outputs
+-------
+
+.. csv-table::
+   :header: "Output Name", "Stream Type"
+
+   "Labels", "Stimulations"
+   "Hyperplane distance", "Streamed matrix"
+   "Probability values", "Streamed matrix"
+
+Labels
+~~~~~~
+
+This output will contain the classification stimulations. Each time a new feature vector arrives to this box,
+a new classification process is triggered, resulting in the generation of the corresponding class stimulation.
+
+Hyperplane distance
+~~~~~~~~~~~~~~~~~~~
+
+This output reflects the classification algorithm status in the form of a matrix of value.  This output will contain one or several distances
+to an hyperplane if the classifier provide it. If not, the matrix will have 0 dimension. The format of this output directly depend on 
+the classification algorithm and of the strategy used by the processor box.
+
+Probability values
+~~~~~~~~~~~~~~~~~~
+
+This output reflects the classification algorithm status in the form of a matrix of value.  This output will contains one or several probabilities
+for a data to be on a class if the classifier provide it. If not, the matrix will have 0 dimension. The format of this output directly depend on 
+the classification algorithm and of the strategy used by the processor box.
+
+.. _Doc_BoxAlgorithm_ClassifierProcessor_Settings:
+
+Settings
+--------
+
+.. csv-table::
+   :header: "Setting Name", "Type", "Default Value"
+
+   "Filename to load configuration from", "Filename", ""
+
+Filename to load configuration from
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This setting points to the configuration file of the box generated by the 
+:ref:`Doc_BoxAlgorithm_ClassifierTrainer` box. Its syntax depends on the selected algorithm.
+
+.. _Doc_BoxAlgorithm_ClassifierProcessor_Examples:
+
+Examples
+--------
+
+This box is used in BCI pipelines in order to classify cerebral activity states. For a detailed scenario using this
+box and its associated :ref:`Doc_BoxAlgorithm_ClassifierTrainer`, please see the **motor imagary**
+BCI scenario in the sample scenarios.
+

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/boxes/Doc_BoxAlgorithm_ClassifierTrainer.rst

@@ -0,0 +1,321 @@
+.. _Doc_BoxAlgorithm_ClassifierTrainer:
+
+Classifier trainer
+==================
+
+.. container:: attribution
+
+   :Author:
+      Yann Renard, Guillaume Serriere
+   :Company:
+      INRIA/IRISA
+
+.. image:: images/Doc_BoxAlgorithm_ClassifierTrainer.png
+
+Performs classifier training with cross-validation -based error estimation
+
+The *Classifier Trainer* box is a generic box for training models to classify input data. 
+It works in conjunction with the :ref:`Doc_BoxAlgorithm_ClassifierProcessor` box.
+This box' role is to expose a generic interface to the rest of the BCI pipelines. The box 
+will generate an internal structure according to the multiclass strategy and the learning
+algorithm selected.
+
+The behavior is simple, the box collects a number of feature vectors. Those feature vectors
+are labelled depending on the input they arrive on. When a specific stimulation arrives, a training
+process is triggered. This process can take some time so this box should be used offline. Depending on the
+settings you enter, you will be able to perform a k-fold test to estimate the accuracy of the learned
+classifier. When this training stimulation is received, the box generates a configuration file that will 
+be usable online by the :ref:`Doc_BoxAlgorithm_ClassifierProcessor` box. 
+Finally, the box outputs a particular stimulation (OVTK_StimulationId_TrainCompleted) 
+on its output, that can be used to trigger further treatments in the scenario.
+
+Inputs
+------
+
+.. csv-table::
+   :header: "Input Name", "Stream Type"
+
+   "Stimulations", "Stimulations"
+   "Features for class 1", "Feature vector"
+   "Features for class 2", "Feature vector"
+
+This box can have a variable number of inputs. If you need more than two classes, feel free to add more
+inputs and to use a proper strategy/classifier combination to handle more than two classes.
+
+Stimulations
+~~~~~~~~~~~~
+
+The first input receives a stimulation stream. Only one stimulation of this stream is important, the one
+that triggers the training process. When this stimulation is received, all the feature vectors are labelled
+and sent to the classification algorithm. The training is triggered and executed. Then the classification
+algorithm generates a configuration file that will be used online by the :ref:`Doc_BoxAlgorithm_ClassifierProcessor` box.
+
+Features for class 1
+~~~~~~~~~~~~~~~~~~~~
+
+This input receives the feature vector for the first class.
+
+Features for class 2
+~~~~~~~~~~~~~~~~~~~~
+
+This input receives the feature vector for the second class.
+
+Outputs
+-------
+
+.. csv-table::
+   :header: "Output Name", "Stream Type"
+
+   "Train-completed Flag", "Stimulations"
+
+Train-completed Flag
+~~~~~~~~~~~~~~~~~~~~
+
+The stimulation OVTK_StimulationId_TrainCompleted is raised on this output when the classifier trainer has finished its job.
+
+.. _Doc_BoxAlgorithm_ClassifierTrainer_Settings:
+
+Settings
+--------
+
+.. csv-table::
+   :header: "Setting Name", "Type", "Default Value"
+
+   "Train trigger", "Stimulation", "OVTK_StimulationId_Train"
+   "Filename to save configuration to", "Filename", "${Path_UserData}/my-classifier.xml"
+   "Multiclass strategy to apply", "Classification strategy", "Native"
+   "Class 1 label", "Stimulation", "OVTK_StimulationId_Label_01"
+   "Class 2 label", "Stimulation", "OVTK_StimulationId_Label_02"
+   "Algorithm to use", "Classification algorithm", "Linear Discrimimant Analysis (LDA)"
+   "Use shrinkage", "Boolean", "false"
+   "Shrinkage coefficient (-1 == auto)", "Float", "-1.000000"
+   "Shrinkage: Force diagonal cov (DDA)", "Boolean", "false"
+   "Number of partitions for k-fold cross-validation test", "Integer", "10"
+   "Balance classes", "Boolean", "false"
+
+The number of settings of this box can vary depending on the classification algorithm you choose. Such algorithm
+could have specific input OpenViBE::Kernel::IParameter objects (see OpenViBE::Kernel::IAlgorithmProxy for details). If
+the type of those parameters is simple enough to be handled in the GUI, then additional settings will be added to this box.
+**After switching a strategy or a classifier, you will have to close and re-open the settings configuration dialog to see the parameters of the new classifier.** Supported parameter types are : Integers, Floats, Enumerations, Booleans. The documentation for those
+parameters can not be done in this page because it is impossible to know at this time what classifier thus what hyper
+parameters you will have available. This will depend on the classification algorihtms that are be implemented in OpenViBE.
+
+Train trigger
+~~~~~~~~~~~~~
+
+The stimulation that triggers the training process and save the learned classifier to disk.
+
+Filename to save configuration to
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This setting points to the configuration file where to save the result of the training for later online use. This
+configuration file is used by the :ref:`Doc_BoxAlgorithm_ClassifierProcessor` box. Its syntax
+depends on the selected algorithm.
+
+Multiclass strategy to apply
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This setting is the strategy to use. You can choose any registered ``OVTK_TypeId_ClassificationStrategy``
+strategy you want.
+
+Class 1 label
+~~~~~~~~~~~~~
+
+This is the stimulation to send when the classifier algorithm detects a class-1 feature vector	
+
+Class 2 label
+~~~~~~~~~~~~~
+
+This is the stimulation to send when the classifier algorithm detects a class-2 feature vector	
+
+Algorithm to use
+~~~~~~~~~~~~~~~~
+
+This setting is the classifier to use. You can choose any registered ``OVTK_TypeId_ClassifierAlgorithm``
+algorithm you want.
+
+Number of partitions for k-fold cross-validation test
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you want to perform a k-fold test, you should enter something else than 0 or 1 here. A k-fold test generally gives
+a better estimate of the classifiers accuracy than naive testing with the training data. The classifier may overfit
+the training data, and get a good accuracy with the observed data, but not be able to generalize to unseen data. 
+In cross-validation, the idea is to divide the set of  feature vectors in a number of partitions. The classification algorithm 
+is trained on some of the partitions and its accuracy is tested on the others. However, the classifier produced by the box is 
+the classifier trained with the whole data. The cross-validation is only an error estimation tool, it does not affect 
+the resulting model. See the miscellaneous section for details on how the k-fold test is done in this box, and possible 
+caveats about the cross-validation procedure.
+
+Balance classes
+~~~~~~~~~~~~~~~
+
+If the number of class labels is unbalanced, the classifiers tend to be biased towards the majority labels.
+This option can be used to resample the dataset to feature all classes equally. 
+
+The algorithm first looks how many examples there are in the majority class. Lets say this is n. Then, if class k has m examples, 
+it will random sample n-m examples with replacement from class k, appending them to the dataset. This will be done for each class. 
+In the end, each class will have n examples and all except the majority class will have some duplicate training vectors. 
+This can be seen as a technique to weight the importance of examples for such classifiers that do not support setting example weights 
+or class weight prior, and can in general be attempted with arbitrary learning algorithms.
+
+Enabling this option may make sense if the box is used for incremental learning, where all classes may not be equally represented 
+in the training data obtained so far, even if the design itself is balanced. Note that enabling this will make the cross-validation 
+results optimistic. In most conditions, the feature should be disabled.
+
+.. _Doc_BoxAlgorithm_ClassifierTrainer_Examples:
+
+Examples
+--------
+
+This box is used in BCI pipelines in order to classify cerebral activity states. For a detailed scenario using this
+box and its associated :ref:`Doc_BoxAlgorithm_ClassifierProcessor`, please see the **motor imagary**
+BCI scenario in the sample scenarios. An even more simple tutorial with artificial data 
+is available in the **box-tutorials/** folder.
+
+.. _Doc_BoxAlgorithm_ClassifierTrainer_Miscellaneous:
+
+Miscellaneous
+-------------
+
+The box supports various multiclass strategies and classifiers as plugins.
+
+\par Available strategy:
+Strategy refers to how feature vectors are routed to one or more classifiers, which possibly can handle only 2 classes themselves.
+
+\par Native
+Use the classifier training algorithm without a pairwise strategy. All the data is passed to a single classifier trainer.
+
+\par One Vs All
+Use a pairwise strategy which consists of training each class against all the others, creating n classifiers for n classes.
+
+\par One vs One
+Use a airwise strategy which trains one classifier for each pair of classes. Then we use a decision startegy to extract the most likely class. There are three differents decision strategy:
+\li Voting: method based on a simple majority voting process
+\li HT: method described in: Hastie, Trevor ; Tibshirani, Robert. Classification by pairwise coupling. The Annals of Statistics 26 (1998), no. 2, 451--471
+\li PKPD: method describe in: Price, S. Knerr, L. Personnaz, and G. Dreyfus. Pairwise neural network classifiers with probabilistic outputs. In G. Tesauro, D. Touretzky, and T. Leen (eds.)
+Advances in Neural Information Processing Systems 7 (NIPS-94), pp. 1109-1116. MIT Press, 1995.
+You cannot use every algorithm with every decision strategy, but the interface will restain the choice according to your selection.
+
+\par Available classifiers:
+
+\par Support Vector Machine (SVM)
+A well-known classifier supporting non-linear classification via kernels. The implementation is based on LIBSVM 2.91, which is included in the OpenViBE source tree. The parameters exposed in the GUI correspond to LIBSVM parameters. For more information on LIBSVM, see <a href="http://www.csie.ntu.edu.tw/~cjlin/libsvm/">here</a>.
+\par 
+This algorithm provides only probabilities.
+
+\par Linear Discriminant Analysis (LDA)
+A simple and fast linear classifier. For description, see any major textbook on Machine Learning or Statistics (e.g. Duda, Hart & Stork, or Hastie, Tibshirani & Friedman). This algorithm can be used with a regularized covariance matrix
+according to a method proposed by Ledoit & Wolf: "A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices", 2004.
+The Linear Discriminant Analysis has the following options.
+\par
+\li Use shrinkage: Use a classic or a regularized covariance matrix.
+\li Shrinkage: A value s between [0,1] sets a linear weight between dataCov and priorCov. I.e. cov=(1-s)\*dataCov+s\*priorCov.
+Value <0 is used to auto-estimate the shrinking coefficient (default). If var(x) is a vector of empirical variances of all data dimensions, priorCov is a 
+diagonal matrix with a single value mean(var(x)) pasted on its diagonal. Used only if use shrinkage is checked.
+\li Force diagonal cov (DDA): This sets the nondiagonal entries of the covariance matrices to zero. Used only if Use shrinkage is checked.
+\par
+Note that setting shrinkage to 0 should get you the regular LDA behavior. If you additionally force the covariance to be diagonal, you should get a model resembling the Naive Bayes classifier.
+\par 
+This algorithm provides both hyperplane distance and probabilities.
+
+\par Multilayer Perceptron (MLP)
+A classifier algorithm which relies on an artificial neural network (<a href="https://hal.inria.fr/inria-00099922/en">Laurent Bougrain. Practical introduction to artificial neural networks. IFAC symposium on automation in Mining, Mineral and Metal Processing - 
+MMM'04, Sep 2004, Nancy, France, 6 p, 2004.</a>). In OpenViBE, the MLP is a 2-layer neural network. The hyperbolic tangent is the activation function of the
+neurons inside the hidden layer. The network is trained using the backpropagation of the gradient. During the training, 80% of the training set is used to compute the gradient, 
+and 20% is used to validate the new model. The different weights and biases are updated only once per iteration (just before the validation). A coefficient alpha (learning coefficient) is used to moderate the importance of 
+the modification of weights and biases to avoid oscillations. The learning stops when the difference of the error per element (computed during validation) of two consecutive iterations is under the value epsilon given as a parameter.
+\par
+\li Number of neurons in hidden layer: number of neurons that will be used in the hidden layer.
+\li Learning stop condition : the epsilon value used to stop the learning
+\li Learning coefficient: a coefficient which influence the speed of learning. The smaller the coefficient is, the longer the learning will take, the more chance you will have to get a good solution.
+\par
+Note that feature vectors are normalized between -1 and 1 (using the min/max of the training set) to avoid saturation of the hyperbolic tangent.
+\par
+This algorithm provides both hyperplane distance (identity of output layer) and probabilites (softmax function on output layer).
+
+\par Cross Validation 
+
+In this section, we will detail how the k-fold test is implemented in this box. For the k-fold test to be performed, you
+have to choose more than 1 partition in the related settings. Suppose you chose ``n`` partitions. Then when trigger stimulation
+is received, the feature vector set is splitted in ``n`` consecutive segments. The classification algorithm is trained on
+``n-1`` of those segments and tested on the last one. This is performed for each segment. 
+
+For example, suppose you have 5 partitions of feature vectors (``FVs)``
+
+.. code::
+
+   +------+ +------+ +------+ +------+ +------+
+   | FVs1 | | FVs2 | | FVs3 | | FVs4 | | FVs5 |
+   +------+ +------+ +------+ +------+ +------+
+
+For the first training, a feature vector set is built form the ``FVs2,`` ``FVs3,`` ``FVs4,`` ``FVs5.`` The classifier algorithm
+is trained on this feature vector set. Then the classifier is tested on the ``FVs1`` :
+
+.. code::
+
+   +------+ +---------------------------------+
+   | FVs1 | |  Training Feature Vector Set 1  |
+   +------+ +---------------------------------+
+
+Then, a feature vector set is built form the ``FVs1,`` ``FVs3,`` ``FVs4,`` ``FVs5.`` The classifier algorithm
+is trained on this feature vector set. Then the classifier is tested on the ``FVs2`` :
+
+.. code::
+
+   +-------+ +------+ +------------------------+
+   | Train | | FVs2 | | ing Feat. Vector Set 2 |
+   +-------+ +------+ +------------------------+
+
+The same process if performed on all the partitions :
+
+.. code::
+
+   +---------------+ +------+ +---------------+
+   |Training Featur| | FVs3 | |e Vector Set 3 |
+   +---------------+ +------+ +---------------+
+   +------------------------+ +------+ +------+
+   |Training Feature Vector | | FVs4 | |Set 4 |
+   +------------------------+ +------+ +------+
+   +---------------------------------+ +------+
+   |  Training Feature Vector Set 5  | | FVs5 |
+   +---------------------------------+ +------+
+
+Important things to consider :
+
+- The more partitions you have, the more feature vectors you have in your training sets... and the less examples 
+
+you'll have to test on. This means that the result of the test will probably be less reliable. 
+
+In conclusion, be careful when choosing this k-fold test setting. Typical value range from 4 partitions (train on 75% of the feature vectors and
+test on 25% - 4 times) to 10 partitions (train on 90% of the feature vectors and test on 10% - 10 times).
+
+Note that the cross-validation performed by the classifier trainer box in OpenViBE may be optimistic.
+The cross-validation computation is working as it should, but it cannot take into account what happens outside
+the classifier trainer box. In OpenViBE scenarios, there may be e.g. time overlap from epoching, feature
+vectors drawn from the same epoch ending up in the same cross-validation partition, and (supervised) 
+preprocessing such as CSP or xDAWN potentially overfitting the data before its given to the classifier trainer. 
+Such situations are not compatible with the theoretical assumption that the feature vectors are 
+independent and identically distributed (the typical iid assumption in machine learning) across
+train and test. To do cross-validation controlling for such issues, we have provided
+a more advanced cross-validation tutorial as part of the OpenViBE web documentation.
+
+\par Confusion Matrices
+
+At the end of the training, the box will print one or two confusion matrices, depending if cross-validation
+was used: one matrix for the cross-validation, the other for the training data. Each matrix will contain true 
+class as rows, and predicted class as columns. The diagonal describes the percentage of correct predictions per class. 
+Although the matrix can be optimistic (see above section about the cross-validation), it may give useful 
+diagnostic information. For example, if the accuracy is very skewed towards one class, this may indicate
+a problem if the design is supposed to be balanced. The problem may originate e.g. from the original data 
+source, the signal processing chains for the different classes, or the classifier learning algorithm. These need
+then to be investigated. Also, if very low accuracies are observed in these matrices, it may give reason 
+to suspect that prediction accuracies on fresh data might be likewise lacking -- or worse.
+
+\par Incremental Learning
+
+The box can also be used for simple incremental (online) learning. To achieve this, simply send the box the training
+stimulation and it will train a classifier with all the data it has received so far. You can give it more
+feature vectors later, and trigger the learning again by sending another stimulation. Likewise, the corresponding 
+classifier processor box can be made to load new classifiers during playback. With classifiers like LDA, 
+this practice is usually feasible when the data is reasonably sized (as in basic motor imagery).
+

Masterarbeit/openvibe/sdk-master/plugins/processing/classification/doc/boxes/Doc_BoxAlgorithm_VotingClassifier.rst

@@ -0,0 +1,132 @@
+.. _Doc_BoxAlgorithm_VotingClassifier:
+
+Voting Classifier
+=================
+
+.. container:: attribution
+
+   :Author:
+      Yann Renard
+   :Company:
+      INRIA
+
+.. image:: images/Doc_BoxAlgorithm_VotingClassifier.png
+
+Each classifier used as input is assumed to have its own two-class output stream. Mainly designed for P300 scenario use.
+
+The purpose of this simple classifier is to choose between multiple two class classifiers which
+one mostly fits a condition. For example imagine ``n`` states. Each of those states can be either active
+or inactive. Additionally, imagine you want only one active state at a time. Then you can have ``n``
+two-class classifiers telling for each state if it is active or not, and a following voting classifier
+that chooses which of those states is the *most active*.
+
+Inputs
+------
+
+.. csv-table::
+   :header: "Input Name", "Stream Type"
+
+   "Classification result 1", "Stimulations"
+   "Classification result 2", "Stimulations"
+
+You can add as many inputs as you need depending on the number of preceeding states.
+
+The inputs of this classifier can be changed to either streamed matrix of stimulations.
+
+In the case you choose stimulations, each active stimulation gives a point
+to the preceeding state. Both an inactive and a reject stimulation gives no point.
+Any other stimulation is ignored.
+After a number of repetitions, the state with the best score is chosen.
+
+In the case you choose streamed matrix, the matrix must have only one
+element. This element is used as a score coefficient (in place of the simple 1-0 of the previous
+case).  After a number of repetitions, the state with the best score is chosen.
+
+Classification result 1
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Input stream for the first state.
+
+Classification result 2
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Input stream for the second state.
+
+Outputs
+-------
+
+.. csv-table::
+   :header: "Output Name", "Stream Type"
+
+   "Classification choice", "Stimulations"
+
+Classification choice
+~~~~~~~~~~~~~~~~~~~~~
+
+This output sends a new stimulation as soon as the classifier received the correct number of votes
+from the preceeding states. The output stimulation is based on the 5th setting of the box. First
+state being selected would send exactly this stimulation. Second state would send this
+stimulation + 1 etc.
+
+.. _Doc_BoxAlgorithm_VotingClassifier_Settings:
+
+Settings
+--------
+
+.. csv-table::
+   :header: "Setting Name", "Type", "Default Value"
+
+   "Number of repetitions", "Integer", "12"
+   "Target class label", "Stimulation", "OVTK_StimulationId_Target"
+   "Non target class label", "Stimulation", "OVTK_StimulationId_NonTarget"
+   "Reject class label", "Stimulation", "OVTK_StimulationId_Label_00"
+   "Result class label base", "Stimulation", "OVTK_StimulationId_Label_01"
+   "Choose one if ex-aequo", "Boolean", "false"
+
+Number of repetitions
+~~~~~~~~~~~~~~~~~~~~~
+
+This setting tells the box how many votes it needs before choosing. If the box uses stimulations,
+it must receive either the target or non target stimulation to consider a state has been voted.
+Additionaly, the box waits each state to be voted the correct number of times to take a decision.
+
+Target class label
+~~~~~~~~~~~~~~~~~~
+
+If the box uses stimulations, this settings tells what stimulation reflects that the state is active.
+
+Non target class label
+~~~~~~~~~~~~~~~~~~~~~~
+
+If the box uses stimulations, this settings tells what stimulation reflects that the state is inactive.
+
+Reject class label
+~~~~~~~~~~~~~~~~~~
+
+In case no choice can be made (for example, there are more than one state with the highest score), the
+voting classifier can choose to reject the vote and send a specific stimulation for this. This is more
+likely to happen when the box works on stimulation better than streamed matrix. You can force this box
+to choose using the 6th setting.
+
+Result class label base
+~~~~~~~~~~~~~~~~~~~~~~~
+
+This stimulation is used as a basis for the stimulations to send when a state is selected. First
+state being selected would send exactly this stimulation. Second state would send this
+stimulation + 1 etc.
+
+Choose one if ex-aequo
+~~~~~~~~~~~~~~~~~~~~~~
+
+This setting can force the box to choose a state even if more than one state have the highest score.
+In such case, there won't be any rejection. The way the box decides between the ex-aequo candidates
+is undefined.
+
+.. _Doc_BoxAlgorithm_VotingClassifier_Examples:
+
+Examples
+--------
+
+This box is used in the *P300 speller* and the *P300 magic card* BCIs. Please see those
+scenarios in the sample *openvibe-scenarios*.
+