Skip to content

Commit

Permalink
Rename files for meta examples (#4055)
Browse files Browse the repository at this point in the history
* Rename files for meta examples
* Update the data submodule version
  • Loading branch information
dgkim5360 authored and karlnapf committed Jan 11, 2018
1 parent e4a19a2 commit 5d1e93d
Show file tree
Hide file tree
Showing 66 changed files with 130 additions and 130 deletions.
2 changes: 1 addition & 1 deletion data
Submodule data updated 31 files
+0 −0 testsuite/meta/binary/averaged_perceptron.dat
+0 −0 testsuite/meta/binary/kernel_support_vector_machine.dat
+0 −0 testsuite/meta/binary/linear_discriminant_analysis.dat
+0 −0 testsuite/meta/binary/linear_support_vector_machine.dat
+0 −0 testsuite/meta/binary/sgd_svm.dat
+0 −0 testsuite/meta/binary/svm_oneclass.dat
+0 −0 testsuite/meta/clustering/gaussian_mixture_models.dat
+0 −0 testsuite/meta/converter/independent_component_analysis_fast.dat
+0 −0 testsuite/meta/converter/independent_component_analysis_ff_sep.dat
+0 −0 testsuite/meta/converter/independent_component_analysis_jade.dat
+0 −0 testsuite/meta/converter/independent_component_analysis_jedi_sep.dat
+0 −0 testsuite/meta/converter/independent_component_analysis_sobi.dat
+0 −0 testsuite/meta/gaussian_process/classifier.dat
+0 −0 testsuite/meta/gaussian_process/regression.dat
+0 −0 testsuite/meta/gaussian_process/sparse_regression.dat
+0 −0 testsuite/meta/multiclass/cartree.dat
+0 −0 testsuite/meta/multiclass/chaid_tree.dat
+0 −0 testsuite/meta/multiclass/ecoc_random.dat
+0 −0 testsuite/meta/multiclass/gaussian_naive_bayes.dat
+0 −0 testsuite/meta/multiclass/gmnp_svm.dat
+0 −0 testsuite/meta/multiclass/k_nearest_neighbours.dat
+0 −0 testsuite/meta/multiclass/large_margin_nearest_neighbours.dat
+0 −0 testsuite/meta/multiclass/linear.dat
+0 −0 testsuite/meta/multiclass/linear_discriminant_analysis.dat
+0 −0 testsuite/meta/multiclass/logistic_regression.dat
+0 −0 testsuite/meta/multiclass/quadratic_discriminant_analysis.dat
+0 −0 testsuite/meta/multiclass/random_forest.dat
+0 −0 testsuite/meta/multiclass/relaxed_tree.dat
+0 −0 testsuite/meta/multiclass/shareboost.dat
+0 −0 testsuite/meta/multiclass/support_vector_machine.dat
+0 −0 testsuite/meta/regression/kernel_ridge_regression_nystrom.dat
Original file line number Diff line number Diff line change
Expand Up @@ -26,27 +26,27 @@ Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CBinaryLabels` as

.. sgexample:: kernel_svm.sg:create_features
.. sgexample:: kernel_support_vector_machine.sg:create_features

In order to run :sgclass:`CLibSVM`, we need to initialize a kernel like :sgclass:`CGaussianKernel` with training features and some parameters like :math:`C` and epsilon i.e. residual convergence parameter which is optional.

.. sgexample:: kernel_svm.sg:set_parameters
.. sgexample:: kernel_support_vector_machine.sg:set_parameters

We create an instance of the :sgclass:`CLibSVM` classifier by passing it regularization coefficient, kernel and labels.

.. sgexample:: kernel_svm.sg:create_instance
.. sgexample:: kernel_support_vector_machine.sg:create_instance

Then we train and apply it to test data, which here gives :sgclass:`CBinaryLabels`.

.. sgexample:: kernel_svm.sg:train_and_apply
.. sgexample:: kernel_support_vector_machine.sg:train_and_apply

We can extract :math:`\alpha` and :math:`b`.

.. sgexample:: kernel_svm.sg:extract_weights_bias
.. sgexample:: kernel_support_vector_machine.sg:extract_weights_bias

Finally, we can evaluate test performance via e.g. :sgclass:`CAccuracyMeasure`.

.. sgexample:: kernel_svm.sg:evaluate_accuracy
.. sgexample:: kernel_support_vector_machine.sg:evaluate_accuracy

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,23 @@ Example

We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CBinaryLabels` from files with training and test data.

.. sgexample:: lda.sg:create_features
.. sgexample:: linear_discriminant_analysis.sg:create_features

We create an instance of the :sgclass:`CLDA` classifier and set features and labels. By default, Shogun automatically chooses the decomposition method based on :math:`{N<=D}` or :math:`{N>D}`.

.. sgexample:: lda.sg:create_instance
.. sgexample:: linear_discriminant_analysis.sg:create_instance

Then we train and apply it to test data, which here gives :sgclass:`CBinaryLabels`.

.. sgexample:: lda.sg:train_and_apply
.. sgexample:: linear_discriminant_analysis.sg:train_and_apply

We can extract weights :math:`{\bf w}`.

.. sgexample:: lda.sg:extract_weights
.. sgexample:: linear_discriminant_analysis.sg:extract_weights

We can evaluate test performance via e.g. :sgclass:`CAccuracyMeasure`.

.. sgexample:: lda.sg:evaluate_accuracy
.. sgexample:: linear_discriminant_analysis.sg:evaluate_accuracy

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,27 +26,27 @@ Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CBinaryLabels` as

.. sgexample:: linear_svm.sg:create_features
.. sgexample:: linear_support_vector_machine.sg:create_features

In order to run :sgclass:`CLibLinear`, we need to initialize some parameters like :math:`C` and epsilon which is the residual convergence parameter of the solver.

.. sgexample:: linear_svm.sg:set_parameters
.. sgexample:: linear_support_vector_machine.sg:set_parameters

We create an instance of the :sgclass:`CLibLinear` classifier by passing it regularization coefficient, features and labels. We here set the solver type to L2 regularized classification. There are many other solver types in :sgclass:`CLibLinear` to choose from.

.. sgexample:: linear_svm.sg:create_instance
.. sgexample:: linear_support_vector_machine.sg:create_instance

Then we train and apply it to test data, which here gives :sgclass:`CBinaryLabels`.

.. sgexample:: linear_svm.sg:train_and_apply
.. sgexample:: linear_support_vector_machine.sg:train_and_apply

We can extract :math:`{\bf w}` and :math:`b`.

.. sgexample:: linear_svm.sg:extract_weights_bias
.. sgexample:: linear_support_vector_machine.sg:extract_weights_bias

We can evaluate test performance via e.g. :sgclass:`CAccuracyMeasure`.

.. sgexample:: linear_svm.sg:evaluate_accuracy
.. sgexample:: linear_support_vector_machine.sg:evaluate_accuracy

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Multiple kernel learning (MKL) is based on convex combinations of arbitrary kern
where :math:`\beta_k > 0`, :math:`\sum_{k=1}^{K} \beta_k = 1`, :math:`K` is the number of sub-kernels, :math:`\bf{k}` is a combined kernel, :math:`{\bf k}_i` is an individual kernel and :math:`{x_i}_i` are the training data.

Classification is done by using Support Vector Machines (SVM). See :doc:`linear_svm` for more details. Optimal :math:`\alpha` and :math:`b` for SVM and :math:`\beta` are determined via training.
Classification is done by using Support Vector Machines (SVM). See :doc:`linear_support_vector_machine` for more details. Optimal :math:`\alpha` and :math:`b` for SVM and :math:`\beta` are determined via training.

See :cite:`sonnenburg2006large` for more details.

Expand All @@ -20,35 +20,35 @@ Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CBinaryLabels` as

.. sgexample:: mkl.sg:create_features
.. sgexample:: multiple_kernel_learning.sg:create_features

Then we create indvidual kernels like :sgclass:`CPolyKernel` and :sgclass:`CGaussianKernel` which will be later combined in one :sgclass:`CCombinedKernel`.

.. sgexample:: mkl.sg:create_kernel
.. sgexample:: multiple_kernel_learning.sg:create_kernel

We create an instance of :sgclass:`CCombinedKernel` and append the :sgclass:`CKernel` objects.

.. sgexample:: mkl.sg:create_combined_train
.. sgexample:: multiple_kernel_learning.sg:create_combined_train

We create an object of :sgclass:`CMKLClassification`, provide the combined kernel and labels before training it.

.. sgexample:: mkl.sg:train_mkl
.. sgexample:: multiple_kernel_learning.sg:train_mkl

After training, we can extract :math:`\beta`, SVM coefficients :math:`\alpha` and :math:`b`.

.. sgexample:: mkl.sg:extract_weights
.. sgexample:: multiple_kernel_learning.sg:extract_weights

We update the :sgclass:`CCombinedKernel` object for testing data.

.. sgexample:: mkl.sg:create_combined_test
.. sgexample:: multiple_kernel_learning.sg:create_combined_test

We set the updated kernel and predict :sgclass:`CBinaryLabels` for test data.

.. sgexample:: mkl.sg:mkl_apply
.. sgexample:: multiple_kernel_learning.sg:mkl_apply

Finally, we can evaluate test performance via e.g. :sgclass:`CAccuracyMeasure`.

.. sgexample:: mkl.sg:evaluate_accuracy
.. sgexample:: multiple_kernel_learning.sg:evaluate_accuracy

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,27 +20,27 @@ Example

We start by creating CDenseFeatures (here 64 bit floats aka RealFeatures) as

.. sgexample:: gmm.sg:create_features
.. sgexample:: gaussian_mixture_models.sg:create_features

We initialize :sgclass:`CGMM`, passing the desired number of mixture components.

.. sgexample:: gmm.sg:create_gmm_instance
.. sgexample:: gaussian_mixture_models.sg:create_gmm_instance

We provide training features to the :sgclass:`CGMM` object, train it by using EM algorithm and sample data-points from the trained model.

.. sgexample:: gmm.sg:train_sample
.. sgexample:: gaussian_mixture_models.sg:train_sample

We extract parameters like :math:`\pi`, :math:`\mu_i` and :math:`\Sigma_i` for any componenet from the trained model.

.. sgexample:: gmm.sg:extract_params
.. sgexample:: gaussian_mixture_models.sg:extract_params

We obtain log likelihood of belonging to clusters and being generated by this model.

.. sgexample:: gmm.sg:cluster_output
.. sgexample:: gaussian_mixture_models.sg:cluster_output

We can also use Split-Merge Expectation-Maximization algorithm :cite:`ueda2000smem` for training.

.. sgexample:: gmm.sg:training_smem
.. sgexample:: gaussian_mixture_models.sg:training_smem

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ Example
Given a dataset which we assume consists of linearly mixed signals, we create CDenseFeatures
(RealFeatures, here 64 bit float values).

.. sgexample:: ica_fast.sg:create_features
.. sgexample:: independent_component_analysis_fast.sg:create_features

We create the :sgclass:`CFastICA` instance, and set its parameter for the iterative optimization.

.. sgexample:: ica_fast.sg:set_parameters
.. sgexample:: independent_component_analysis_fast.sg:set_parameters

Then we apply ICA, which gives the unmixed signals.

.. sgexample:: ica_fast.sg:apply_convert
.. sgexample:: independent_component_analysis_fast.sg:apply_convert

We can also extract the estimated mixing matrix.

.. sgexample:: ica_fast.sg:extract
.. sgexample:: independent_component_analysis_fast.sg:extract

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,39 +15,39 @@ We'll use as example a classification problem solvable by using :sgclass:`CMKLCl
For the sake of brevity, we'll skip the initialization of features, kernels and so on
(see :doc:`../regression/multiple_kernel_learning` for a more complete example of MKL usage).

.. sgexample:: cross_validation_mkl_weight_storage.sg:create_classifier
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:create_classifier

Firstly, we initialize a splitting strategy :sgclass:`CStratifiedCrossValidationSplitting`, which is needed
to divide the dataset into folds, and an evaluation criterium :sgclass:`CAccuracyMeasure`, to evaluate the
performance of the trained models. Secondly, we create the :sgclass:`CCrossValidation` instance.
We set also the number of cross validation's runs.

.. sgexample:: cross_validation_mkl_weight_storage.sg:create_cross_validation
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:create_cross_validation

To observe also the partial folds' results, we create a cross validation's observer :sgclass:`CParameterObserverCV`
and then we register it into the :sgclass:`CCrossValidation` instance.

.. sgexample:: cross_validation_mkl_weight_storage.sg:create_observer
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:create_observer

Finally, we evaluate the model and get the results (aka a :sgclass:`CCrossValidationResult` instance).

.. sgexample:: cross_validation_mkl_weight_storage.sg:evaluate_and_get_result
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:evaluate_and_get_result

We get the :math:`mean` of all the evaluation results and its standard deviation :math:`stddev`.

.. sgexample:: cross_validation_mkl_weight_storage.sg:get_results
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:get_results

We can get more information about the single cross validation's runs and folds by using the observer we set before, like the kernels' weights.
We get the :sgclass:`CMKLClassification` machine used during the first run and trained on the first fold.

.. sgexample:: cross_validation_mkl_weight_storage.sg:get_fold_machine
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:get_fold_machine

Then, from the trained machine, we get the weights :math:`\mathbf{w}` of its kernels.

.. sgexample:: cross_validation_mkl_weight_storage.sg:get_weights
.. sgexample:: cross_validation_multiple_kernel_learning_weights_storage.sg:get_weights

----------
References
----------

:wiki:`Cross-validation_(statistics)`
:wiki:`Cross-validation_(statistics)`
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Gaussian Process Classifier

Application of Gaussian processes in binary and multi-class classification.
`See Gaussian process regression cookbook
<http://shogun.ml/cookbook/latest/examples/gaussian_processes/gaussian_process_regression.html>`_
<http://shogun.ml/cookbook/latest/examples/gaussian_process/regression.html>`_
and :cite:`Rasmussen2005GPM` for more information on Gaussian processes.

-------
Expand All @@ -13,12 +13,12 @@ Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and :sgclass:`CMulticlassLabels` as

.. sgexample:: gaussian_process_classifier.sg:create_features
.. sgexample:: classifier.sg:create_features

To fit the input (training) data :math:`\mathbf{X}`, we have to choose appropriate :sgclass:`CMeanFunction`
and :sgclass:`CKernel`. Here we use a basic :sgclass:`CConstMean` and a :sgclass:`CGaussianKernel` with chosen width parameter.

.. sgexample:: gaussian_process_classifier.sg:create_appropriate_kernel_and_mean_function
.. sgexample:: classifier.sg:create_appropriate_kernel_and_mean_function

We need to specify the inference method to find the posterior distribution of the function values :math:`\mathbf{f}`.
Here we choose to perform Laplace approximation inference method with an instance of :sgclass:`CMultiLaplaceInferenceMethod` (See Chapter 18.2 in :cite:`barber2012bayesian` for a detailed introduction)
Expand All @@ -27,20 +27,20 @@ the training features, the mean function, the labels and an instance of :sgclass
to specify the distribution of the targets/labels as above.
Finally we create an instance of the :sgclass:`CGaussianProcessClassification` classifier.

.. sgexample:: gaussian_process_classifier.sg:create_instance
.. sgexample:: classifier.sg:create_instance

Then we can train the model and evaluate the predictive distribution.
We get predicted :sgclass:`CMulticlassLabels`.

.. sgexample:: gaussian_process_classifier.sg:train_and_apply
.. sgexample:: classifier.sg:train_and_apply

We can extract the probabilities:

.. sgexample:: gaussian_process_classifier.sg:extract_the_probabilities
.. sgexample:: classifier.sg:extract_the_probabilities

We can evaluate test performance via e.g. :sgclass:`CMulticlassAccuracy`.

.. sgexample:: gaussian_process_classifier.sg:evaluate_accuracy
.. sgexample:: classifier.sg:evaluate_accuracy

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,32 @@ Example

Imagine we have files with training and test data. We create `CDenseFeatures` (here 64 bit floats aka RealFeatures) and :sgclass:`CRegressionLabels` as:

.. sgexample:: gaussian_process_regression.sg:create_features
.. sgexample:: regression.sg:create_features

To fit the input (training) data :math:`\mathbf{X}`, we have to choose an appropriate :sgclass:`CMeanFunction` and :sgclass:`CKernel` and instantiate them. Here we use a basic :sgclass:`CZeroMean` and a :sgclass:`CGaussianKernel` with chosen width parameter.

.. sgexample:: gaussian_process_regression.sg:create_appropriate_kernel_and_mean_function
.. sgexample:: regression.sg:create_appropriate_kernel_and_mean_function

We need to specify the inference method to find the posterior distribution of the function values :math:`\mathbf{f}`. Here we choose to perform exact inference with an instance of :sgclass:`CExactInferenceMethod` and pass it the chosen kernel, the training features, the mean function, the labels and an instance of :sgclass:`CGaussianLikelihood`, to specify the distribution of the targets/labels as above. Finally we generate a CGaussianProcessRegression class to be trained.

.. sgexample:: gaussian_process_regression.sg:create_instance
.. sgexample:: regression.sg:create_instance

Then we can train the model and evaluate the predictive distribution. We get predicted :sgclass:`CRegressionLabels`.

.. sgexample:: gaussian_process_regression.sg:train_and_apply
.. sgexample:: regression.sg:train_and_apply

We can compute the predictive variances as

.. sgexample:: gaussian_process_regression.sg:compute_variance
.. sgexample:: regression.sg:compute_variance

The prediction above is based on arbitrarily set hyperparameters :math:`\boldsymbol{\theta}`: kernel width :math:`\tau`, kernel scaling :math:`\gamma` and observation noise :math:`\sigma^2`. We can also learn these parameters by optimizing the marginal likelihood :math:`p(\mathbf{y}|\mathbf{X}, \boldsymbol{\theta})` w.r.t. :math:`\boldsymbol{\theta}`.
To do this, we define a :sgclass:`CGradientModelSelection`, passing to it a :sgclass:`CGradientEvaluation` with its own :sgclass:`CGradientCriterion`, specifying the gradient scheme and direction. Then we can follow the gradient and apply the chosen :math:`\boldsymbol{\theta}` back to the CGaussianProcessRegression instance.

.. sgexample:: gaussian_process_regression.sg:optimize_marginal_likelihood
.. sgexample:: regression.sg:optimize_marginal_likelihood

Finally, we evaluate the :sgclass:`CMeanSquaredError` and the (negative log) marginal likelihood for the optimized hyperparameters.

.. sgexample:: gaussian_process_regression.sg:evaluate_error_and_marginal_likelihood
.. sgexample:: regression.sg:evaluate_error_and_marginal_likelihood

----------
References
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,31 +18,31 @@ Example

Imagine we have files with training and test data. We create `CDenseFeatures` (here 64 bit floats aka RealFeatures) and :sgclass:`CRegressionLabels` as:

.. sgexample:: sparse_gaussian_process_regression.sg:create_features
.. sgexample:: sparse_regression.sg:create_features

To fit the input (training) data :math:`\mathbf{X}`, we have to choose an appropriate :sgclass:`CMeanFunction` and :sgclass:`CKernel` and instantiate them. Here we use a basic :sgclass:`CZeroMean` and a :sgclass:`CGaussianKernel` with chosen width parameter.

.. sgexample:: sparse_gaussian_process_regression.sg:create_kernel_and_mean_function
.. sgexample:: sparse_regression.sg:create_kernel_and_mean_function

We need to specify the inference method to find the posterior distribution of the function values :math:`\mathbf{f}`. Here we choose to perform variational inference for fully independent conditional training (FITC) with an instance of :sgclass:`CFITCInferenceMethod`. We use another feature instance for inducing points and add a simple subset for demonstration. The inference method is then created and we pass it the chosen kernel, the training features, the mean function, the labels, an instance of :sgclass:`CGaussianLikelihood`. We use a subset of the training data for inducing features.

.. sgexample:: sparse_gaussian_process_regression.sg:create_inference
.. sgexample:: sparse_regression.sg:create_inference

Finally we generate a :sgclass:`CGaussianProcessRegression` class to be trained.

.. sgexample:: sparse_gaussian_process_regression.sg:create_instance
.. sgexample:: sparse_regression.sg:create_instance

Then we can train the model and evaluate the predictive distribution. We get predicted :sgclass:`CRegressionLabels`.

.. sgexample:: sparse_gaussian_process_regression.sg:train_and_apply
.. sgexample:: sparse_regression.sg:train_and_apply

We can compute the predictive variances as

.. sgexample:: sparse_gaussian_process_regression.sg:compute_variance
.. sgexample:: sparse_regression.sg:compute_variance

Finally, we evaluate the :sgclass:`CMeanSquaredError`.

.. sgexample:: sparse_gaussian_process_regression.sg:evaluate_error
.. sgexample:: sparse_regression.sg:evaluate_error

----------
References
Expand Down

0 comments on commit 5d1e93d

Please sign in to comment.