Machine Learning Algorithms

Here is a list of the machine learning algorithms available in the Foundry. These algorithms are all implemented in Java and are designed to be used in applications and extended for research.

Supervised Learning

Supervised learning algorithms take input-output pairs to train a function that attempts generalize to produce outputs for new and unseen inputs.

Batch Supervised Learning

• Naive Bayes
• AdaBoost
• Bagging and Balanced Bagging
• Decision Tree
• IVoting and Balanced IVoting
• Nearest Neighbor
• K-Nearest Neighbor
• Kernel Regression
• Linear Regression and Multivariate Linear Regression
• Locally-weighted Regression
• Logistic Regression
• Perceptron
• Random Forests
• Regression Tree
• Robust Regression
• Support Vector Machine via Sequential Minimal Optimization (SMO), Successive Overrelaxation, and Primal Estimated Sub-Gradient Solver (PEGASOS) with a variety of kernels

Incremental Supervised Learning

These are supervised algorithms that can learn incrementally from a stream of data, commonly called online learning. Many have both linear and kernel forms.

• Adaptive Regularization of Weights (AROW)
• Adatron
• Aggressive Relaxed Online Maximum Margin Algorithm (AROMMA)
• Ballseptron
• Confidence Weighted Linear Classification
• Forgetron
• Margin Infused Relaxed Algorithm (MIRA)
• Online Bagging
• Online Perceptron
• Passive-Aggressive Perceptron (PA-I and PA-II)
• Projectron
• Ramp Loss Passive-Aggressive Perceptron (PA^R)
• Shifting Perceptron
• Stoptron
• Online Voted Perceptron
• Relaxed Online Maximum Margin Algorithm (ROMMA)
• Winnow

Unsupervised Learning

The unsupervised learning algorithms are used with data that is not labeled. Clustering algorithms are usually dependent on using a provided distance metric.

• Affinity Propagation
• Hierarchical Agglomerative Clustering
• Dirichlet Process Clustering
• [K-Means Clustering] (https://github.com/algorithmfoundry/Foundry/blob/master/Components/LearningCore/Source/gov/sandia/cognition/learning/algorithm/clustering/KMeansClusterer.java)
• Partitional Custering
• Hidden Markov Model
• Markov Chain
• Thin Singular Value Decomposition
• Principal Components Analysis (PCA)
• Kernel Principal Components Analysis
• Generalized Hebbian Algorithm

Optimization Algorithms for Learning

General optimization methods can usually work with a variety of learned function types. A common one would be a Generalized Linear Model (GLM) or Neural Network that can be used with various activation functions.

• Broyden-Fletcher-Goldfarb-Shanno (BFGS)
• Conjugate Gradient
• Fletcher Xu Hybrid Estimation
• Davidon-Flecher-Powell (DFP)
• Direction Set (Powell's Method)
• Fletcher-Revees conjugate gradient
• Gauss-Newton
• Gradient Descent
• Least-squares
• Levenberg Marquardt
• Liu-Storey conjugate gradient
• Downhill Simplex (Nelder-Mead)
• Polack-Ribiere conjugate gradient
• Root Finding
• Simulated Annealing
• Genetic Algorithms

Baseline Learners

These are just simple baseline learners that can be compared against.

• Mean
• Most Frequent