`Perceptron`

The Perceptron class implements the simple perceptron classifier originally implemented by Frank Rosenblatt in 1958. The perceptron is a linear classifier, and can be understood as a trivial neural network with one neuron that uses the step function as an activation function. mlpack's implementation of the Perceptron class also offers several template parameters that can be used to control the behavior of the perceptron.

Perceptrons are useful for classifying points with discrete labels (i.e., 0, 1, 2). Because they are simple classifiers, they are also useful as weak learners for the AdaBoost boosting classifier.

Simple usage example:

// Train a perceptron on random numeric data and predict labels on test data:

// All data and labels are uniform random; 10 dimensional data, 5 classes.
// Replace with a data::Load() call or similar for a real application.
arma::mat dataset(10, 1000, arma::fill::randu);
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));
arma::mat testDataset(10, 500, arma::fill::randu); // 500 test points.

mlpack::Perceptron p;                 // Step 1: create model.
p.Train(dataset, labels, 5);          // Step 2: train model.
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions); // Step 3: classify points.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;

More examples...

Quick links:

Constructors: create DecisionTree objects.
Train(): train model.
Classify(): classify with a trained model.
Other functionality for loading, saving, and inspecting.
Examples of simple usage and links to detailed example projects.
Template parameters for custom behavior.
Advanced template examples of use with custom template parameters.

Constructors

Construct a Perceptron object using one of the constructors below. Defaults and types are detailed in the Constructor Parameters section below.

Forms:

p = Perceptron()
- Initialize perceptron without training.
- You will need to call Train() later to train the perceptron before calling Classify().

p = Perceptron(numClasses, dimensionality, maxIterations=1000)
- Initialize perceptron with all-zero weights and biases.
- Classify() can immediately be used; training is not required with this form.

p = Perceptron(data, labels, numClasses, maxIterations=1000)
p = Perceptron(data, labels, numClasses, weights, maxIterations=1000)
- Train the perceptron (optionally with instance weights).

Constructor Parameters:

name	type	description	default
`data`	`arma::mat`	Column-major training matrix.	(N/A)
`datasetInfo`	`data::DatasetInfo`	Dataset information, specifying type information for each dimension.	(N/A)
`labels`	`arma::Row<size_t>`	Training labels, between `0` and `numClasses - 1` (inclusive). Should have length `data.n_cols`.	(N/A)
`weights`	`arma::rowvec`	Weights for each training point. Should have length `data.n_cols`.	(N/A)
`numClasses`	`size_t`	Number of classes in the dataset.	(N/A)
`dimensionality`	`size_t`	Dimensionality of data (only used if an initialized but untrained model is desired).	(N/A)
`maxIterations`	`size_t`	Maximum number of iterations during training. Can also be set with `MaxIterations()`.	`1000`

As an alternative to passing maxIterations, it can be set with a standalone method. The following function can be used before calling Train() to set the maximum number of iterations:

p.MaxIterations() = maxIter; will set the maximum number of iterations during training to maxIter.

Training

If training is not done as part of the constructor call, it can be done with one of the following versions of the Train() member function:

p.Train(data, labels, numClasses, maxIterations=1000)
- Train the perceptron on unweighted data.

p.Train(data, labels, numClasses, weights, maxIterations=1000)
- Train the perceptron on data with instance weights.

Types of each argument are the same as in the table for constructors above.

Notes:

Training is incremental. Successive calls to Train() will not reinitialize the model, unless the given data has different dimensionality or numClasses is different. To reinitialize the model, call Reset() (see Other Functionality).
If maxIterations is not passed, but has been set in the constructor or with MaxIterations(), the previous setting will be used.

Classification

Once a Perceptron is trained, the Classify() member function can be used to make class predictions for new data.

size_t predictedClass = p.Classify(point)
- (Single-point)
- Classify a single point, returning the predicted class.

p.Classify(data, predictions)
- (Multi-point)
- Classify a set of points.
- The prediction for data point i can be accessed with predictions[i].

Note: perceptrons do not provide any measure resembling probabilities during classification, and thus a version of Classify() that computes class probabilities is not available.

Classification Parameters:

usage	name	type	description
single-point	`point`	`arma::vec`	Single point for classification.

multi-point	`data`	`arma::mat`	Set of column-major points for classification.
multi-point	`predictions`	`arma::Row<size_t>&`	Vector of `size_t`s to store class prediction into. Will be set to length `data.n_cols`.

Other Functionality

A Perceptron can be serialized with data::Save() and data::Load().
p.NumClasses() will return a size_t indicating the number of classes the perceptron was trained on.
p.Biases() will return an arma::vec with the biases of the model (each element corresponds to the bias for a class).
p.Weights() will return an arma::mat with the weights of the model (each column corresponds to the weights for one class label).
p.Reset() will re-initialize the weights and biases of the model.

For complete functionality, the source code can be consulted. Each method is fully documented.

Simple Examples

See also the simple usage example for a trivial use of Perceptron.

Train a perceptron multiple times, incrementally, with custom hyperparameters, and save the resulting model to disk.

// See https://datasets.mlpack.org/iris.csv.
arma::mat dataset;
mlpack::data::Load("iris.csv", dataset, true);
// See https://datasets.mlpack.org/iris.labels.csv.
arma::Row<size_t> labels;
mlpack::data::Load("iris.labels.csv", labels, true);

// Create a Perceptron object.
mlpack::Perceptron p;
// Set the maximum number of iterations to 100.  (This can also be done in the
// constructor.)
p.MaxIterations() = 100;

// Train the model for up to 100 iterations.
p.Train(dataset, labels, 3);

// Now, compute and print accuracy on the training set.
arma::Row<size_t> predictions;
p.Classify(dataset, predictions);
std::cout << "Training set accuracy after 100 iterations: "
    << (100.0 * double(arma::accu(labels == predictions)) / labels.n_elem)
    << "\%." << std::endl;

// Train for another 250 iterations and compute training set accuracy again.
p.Train(dataset, labels, 3, 250);
p.Classify(dataset, predictions);
std::cout << "Training set accuracy after 350 iterations: "
    << (100.0 * double(arma::accu(labels == predictions)) / labels.n_elem)
    << "\%." << std::endl;

// Save the perceptron to disk for later use.
mlpack::data::Save("perceptron.bin", "perceptron", p);

Load a saved perceptron from disk and print information about it.

mlpack::Perceptron p;
// This call assumes a perceptron called "p" has already been saved to
// `perceptron.bin` with `data::Save()`.
mlpack::data::Load("perceptron.bin", "p", p, true);

if (p.NumClasses() > 0)
{
  std::cout << "The perceptron in `perceptron.bin` was trained on "
      << p.NumClasses() << " classes." << std::endl;
  std::cout << "The dimensionality of the perceptron model is "
      << p.Weights().n_rows << "." << std::endl;
  std::cout << "The bias weights for each class are:" << std::endl;
  for (size_t i = 0; i < p.NumClasses(); ++i)
    std::cout << "  - Class " << i << ": " << p.Biases()[i] << std::endl;
}
else
{
  std::cout << "The perceptron in `perceptron.bin` has not been trained."
      << std::endl;
}

Advanced Functionality: Template Parameters

The Perceptron class also supports several template parameters, which can be used for custom behavior. The full signature of the class is as follows:

Perceptron<LearnPolicy,
           WeightInitializationPolicy,
           MatType>

LearnPolicy: the strategy used to learn the weights during training.
WeightInitializationPolicy: the way that weights are initialized before training.
MatType: specifies the type of matrix used for learning and internal representation of weights and biases.

`LearnPolicy`

Specifies the step to be taken when a point is misclassified.
The SimpleWeightUpdate class is available, and is the default.
A custom class must implement only one function:

// You can use this as a starting point for implementation.
class CustomLearnPolicy
{
  // Update the weights and biases in the `weights` matrix and the `biases`
  // vector given that the model currently classified `trainingPoint` as having
  // the label `incorrectClass`, when in reality it has the label
  // `correctClass`.  If `instanceWeight` is given, it specifies the instance
  // weight for the given `trainingPoint`.
  //
  // `VecType` will be an Armadillo-like vector type.  It will be a column from
  // the training data matrix (`data`) given to `Train()` or to the constructor.
  //
  // `eT` is the element type of the Perceptron (e.g. `float`, `double`).
  template<typename VecType, typename eT>
  void UpdateWeights(const VecType& trainingPoint,
                     arma::Mat<eT>& weights,
                     arma::Col<eT>& biases,
                     const size_t incorrectClass,
                     const size_t correctClass,
                     const double instanceWeight = 1.0);
};

`WeightInitializationPolicy`

Specifies how the weights matrix and biases vector should be initialized when the Perceptron object is created, or when Reset() is called.
The ZeroInitialization (default) and RandomPerceptronInitialization classes are available for drop-in usage.
RandomPerceptronInitialization will initialize weights and biases using a uniform random distribution between 0 and 1.
A custom class must implement only one function:

// You can use this as a starting point for implementation.
class CustomWeightInitializationPolicy
{
  // Initialize the `weights` matrix and `biases` vector, given that the model
  // will have dimensionality of `numFeatures` (that is, the training data
  // matrix will have `numFeatures` rows), and the training data has
  // `numClasses` classes.
  //
  // The initialized `weights` matrix should have `numFeatures` rows and
  // `numClasses` columns, and the initialized `biases` vector should have
  // `numClasses` elements.
  //
  // `eT` specifies the element type of the weights and biases; it may be
  // `double`, `float`, or another floating-point type.
  template<typename eT>
  inline static void Initialize(arma::Mat<eT>& weights,
                                arma::Col<eT>& biases,
                                const size_t numFeatures,
                                const size_t numClasses)
  {
    weights.randu(numFeatures, numClasses);
    biases.randu(numClasses);
  }
};

`MatType`

Specifies the matrix type to use for data when learning a perceptron.
By default, MatType is arma::mat (dense 64-bit precision matrix).
Any matrix type implementing the Armadillo API will work; so, for instance, arma::fmat or arma::sp_mat can be used.

Advanced Functionality Examples

Train a Perceptron with random initialization, instead of zero initialization of weights.

// 1000 random points in 10 dimensions.
arma::mat dataset(10, 1000, arma::fill::randu);
// Random labels for each point, totaling 5 classes.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));

// Train in the constructor.  Weights will be initialized randomly.
mlpack::Perceptron<mlpack::SimpleWeightUpdate,
                   mlpack::RandomPerceptronInitialization> p(
    dataset, labels, 5);

// Create test data (500 points).
arma::mat testDataset(10, 500, arma::fill::randu);
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions);
// Now `predictions` holds predictions for the test dataset.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;

Train a Perceptron on sparse 32-bit floating point data.

// 1000 sparse random points in 100 dimensions, with 1% nonzero elements.
arma::sp_fmat dataset;
dataset.sprandu(100, 1000, 0.01);
// Random labels for each point, totaling 5 classes.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));

// Train in the constructor.
mlpack::Perceptron p(dataset, labels, 5);

// Create test data (500 points).
arma::sp_fmat testDataset;
testDataset.sprandu(100, 500, 0.01);
arma::Row<size_t> predictions;
p.Classify(testDataset, predictions);
// Now `predictions` holds predictions for the test dataset.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 1) << " test points classified as class "
    << "1." << std::endl;

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perceptron.md

perceptron.md

`Perceptron`

Simple usage example:

Quick links:

See also:

Constructors

Forms:

Constructor Parameters:

Training

Classification

Classification Parameters:

Other Functionality

Simple Examples

Advanced Functionality: Template Parameters

`LearnPolicy`

`WeightInitializationPolicy`

`MatType`

Advanced Functionality Examples

Files

perceptron.md

Latest commit

History

perceptron.md

File metadata and controls

Perceptron

Simple usage example:

Quick links:

See also:

Constructors

Forms:

Constructor Parameters:

Training

Classification

Classification Parameters:

Other Functionality

Simple Examples

Advanced Functionality: Template Parameters

LearnPolicy

WeightInitializationPolicy

MatType

Advanced Functionality Examples

`Perceptron`

`LearnPolicy`

`WeightInitializationPolicy`

`MatType`