Machine Learning Algorithms

Here is a list of the machine learning algorithms available in the Foundry

Supervised Learning

Supervised learning algorithms take input-output pairs to train a function that attempts generalize to produce outputs for new and unseen inputs.

Batch Supervised Learning

• Naive Bayes
• AdaBoost
• Bagging and Balanced Bagging
• Decision Tree
• IVoting and Balanced IVoting
• Nearest Neighbor
• K-Nearest Neighbor
• Kernel Regression
• Linear Regression and Multivariate Linear Regression
• Locally-weighted Regression
• Logistic Regression
• Perceptron
• Random Forests
• Regression Tree
• Robust Regression
• Support Vector Machine via Sequential Minimal Optimization (SMO), Successive Overrelaxation, and Primal Estimated Sub-Gradient Solver (PEGASOS) with a variety of kernels

Incremental Supervised Learning

These are supervised algorithms that can learn incrementally from a stream of data, commonly called online learning. Many have both linear and kernel forms.

• Adaptive Regularization of Weights (AROW)
• Adatron
• Aggressive Relaxed Online Maximum Margin Algorithm (AROMMA)
• Ballseptron
• Confidence Weighted Linear Classification
• Forgetron
• Margin Infused Relaxed Algorithm (MIRA)
• Online Bagging
• Online Perceptron
• Passive-Aggressive Perceptron (PA-I and PA-II)
• Projectron
• Ramp Loss Passive-Aggressive Perceptron (PA^R)
• Shifting Perceptron
• Stoptron
• Online Voted Perceptron
• Relaxed Online Maximum Margin Algorithm (ROMMA)
• Winnow

Unsupervised Learning

The unsupervised learning algorithms are used with data that is not labeled. Clustering algorithms are usually dependent on using a provided distance metric.

• Affinity Propagation
• Hierarchical Agglomerative Clustering
• Dirichlet Process Clustering
• [K-Means Clustering] (https://github.com/algorithmfoundry/Foundry/blob/master/Components/LearningCore/Source/gov/sandia/cognition/learning/algorithm/clustering/KMeansClusterer.java)
• Partitional Custering
• Hidden Markov Model
• Thin Singular Value Decomposition
• Principal Components Analysis (PCA)
• Kernel Principal Components Analysis
• Generalized Hebbian Algorithm

Optimization Algorithms for Learning

General optimization methods can usually work with a variety of learned function types. A common one would be a Generalized Linear Model (GLM) or Neural Network that can be used with various activation functions.

• BFGS
• Conjugate Gradient
• Fletcher Xu Hybrid Estimation
• Davidon-Flecher-Powell (DFP)
• Direction Set (Powell's Method)
• Fletcher-Revees conjugate gradient
• Gauss-Newton
• Gradient Descent
• Least-squares
• Levenberg Marquardt
• Liu-Storey conjugate gradient
• Downhill Simplex (Nelder-Mead)
• Polack-Ribiere conjugate gradient
• Root Finding
• Simulated Annealing
• Genetic Algorithms

Baseline Learners

These are just simple baseline learners that can be compared against.

• Mean
• Most Frequent