cookbook: multiclass logistic regression

doc/cookbook/source/examples/classifier/multiclass_logisticregression.rst

@@ -0,0 +1,50 @@
+===============================
+Multi-class Logistic Regression
+===============================
+
+Multinomial logistic regression assigns the sample :math:`\mathbf{x}_i` to class :math:`c`
+based on the probability for sample :math:`\mathbf{x}_i` to be in class :math:`c`:
+
+.. math::
+
+    P(Y_i = c | \mathbf{x}_i) = \frac{\exp(\mathbf{\theta}^\top_c\mathbf{x}_i)}{1+ \sum_{k=1}^{K}\exp(\mathbf{\theta}^\top_k\mathbf{x}_i)}
+
+in which :math:`K` is the number of classes.
+
+The loss function that needs to be minimized is:
+
+.. math::
+
+    {\min_{\mathbf{\theta}}}\sum_{k=1}^{K}\sum_{i=1}^{m}w_{ik}\log(1+\exp(-y_{ik}(\mathbf{x}_k^\top\mathbf{a}_{ik} + c_k))) + \lambda\left \| \mathbf{x} \right \|_{l_1/l_q}
+
+where :math:`\mathbf{a}_{ik}` denotes the :math:`i`-th sample for the :math:`k`-th class, :math:`w_{ik}` is the weight for :math:`\mathbf{a}_{ik}^\top`,
+:math:`y_{ik}` is the response of :math:`\mathbf{a}_{ik}`, and :math:`c_k` is the intercept (scalar) for the :math:`k`-th class. 
+:math:`\lambda` is the :math:`l_1/l_q`-norm regularization parameter.
+
+-------
+Example
+-------
+
+Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CMulticlassLabels` as
+
+.. sgexample:: multiclass_logisticregression.sg:create_features
+
+We create an instance of the :sgclass:`CMulticlassLogisticRegression` classifier by passing it the dataset, lables, and specifying the regularization constant :math:`\lambda` for each machine
+
+.. sgexample:: multiclass_logisticregression.sg:create_instance
+
+Then we train and apply it to test data, which here gives :sgclass:`CMulticlassLabels`.
+
+.. sgexample:: multiclass_logisticregression.sg:train_and_apply
+
+We can evaluate test performance via e.g. :sgclass:`CMulticlassAccuracy`.
+
+.. sgexample:: multiclass_logisticregression.sg:evaluate_accuracy
+
+----------
+References
+----------
+
+:wiki:`Multinomial_logistic_regression`
+
+:wiki:`Multiclass_classification`

examples/meta/src/classifier/multiclass_logisticregression.sg

@@ -0,0 +1,29 @@
+CSVFile f_feats_train("../../data/classifier_4class_2d_linear_features_train.dat")
+CSVFile f_feats_test("../../data/classifier_4class_2d_linear_features_test.dat")
+CSVFile f_labels_train("../../data/classifier_4class_2d_linear_labels_train.dat")
+CSVFile f_labels_test("../../data/classifier_4class_2d_linear_labels_test.dat")
+
+#![create_features]
+RealFeatures features_train(f_feats_train)
+RealFeatures features_test(f_feats_test)
+MulticlassLabels labels_train(f_labels_train)
+MulticlassLabels labels_test(f_labels_test)
+#![create_features]
+
+
+#![create_instance]
+MulticlassLogisticRegression classifier(1, features_train, labels_train)
+#![create_instance]
+
+#![train_and_apply]
+classifier.train()
+MulticlassLabels labels_predict = classifier.apply_multiclass(features_test)
+#![train_and_apply]
+
+#![evaluate_accuracy]
+MulticlassAccuracy eval()
+real accuracy = eval.evaluate(labels_predict, labels_test)
+#![evaluate_accuracy]
+
+# integration testing variables
+RealVector output = labels_predict.get_labels()