Machine Learning Algorithms

Here is a list of the machine learning algorithms available in the Foundry. These algorithms are all implemented in Java and are designed to be used in applications and extended for research.

Supervised Learning

Supervised learning algorithms take input-output pairs to train a function that attempts generalize to produce outputs for new and unseen inputs.

Batch Supervised Learning

• AdaBoost
• Bagging and Balanced Bagging
• Decision Tree
• IVoting and Balanced IVoting
• Kernel Regression
• Naive Bayes
• Nearest Neighbor andK-Nearest Neighbor
• Linear Regression and Multivariate Linear Regression
• Locally-weighted Regression
• Logistic Regression
• Perceptron
• Random Forests
• Regression Tree
• Robust Regression
• Support Vector Machine via Sequential Minimal Optimization (SMO), Successive Overrelaxation, and Primal Estimated Sub-Gradient Solver (PEGASOS) with a variety of kernels

Incremental Supervised Learning

These are supervised algorithms that can learn incrementally from a stream of data, commonly called online learning. Many have both linear and kernel forms.

• Adaptive Regularization of Weights (AROW)
• Adatron
• Aggressive Relaxed Online Maximum Margin Algorithm (AROMMA)
• Ballseptron
• Confidence Weighted Linear Classification
• Forgetron
• Margin Infused Relaxed Algorithm (MIRA)
• Online Bagging
• Online Perceptron
• Online Voted Perceptron
• Passive-Aggressive Perceptron (PA-I and PA-II)
• Projectron
• Ramp Loss Passive-Aggressive Perceptron (PA^R)
• Relaxed Online Maximum Margin Algorithm (ROMMA)
• Shifting Perceptron
• Stoptron
• Winnow

Unsupervised Learning

The unsupervised learning algorithms are used with data that is not labeled. Clustering algorithms are usually dependent on using a provided distance metric.

• Affinity Propagation
• Dirichlet Process Clustering
• Generalized Hebbian Algorithm
• Hidden Markov Model
• Hierarchical Agglomerative Clustering
• K-Means Clustering and Generalized Expectation Maximization Hard Assignment Clustering
• Partitional Custering
• Markov Chain
• Principal Components Analysis (PCA) and Kernel Principal Components Analysis
• Thin Singular Value Decomposition

Optimization Algorithms for Learning

General optimization methods can usually work with a variety of learned function types. A common one would be a Generalized Linear Model (GLM) or Neural Network that can be used with various activation functions.

• Broyden-Fletcher-Goldfarb-Shanno (BFGS)
• Conjugate Gradient
• Davidon-Flecher-Powell (DFP)
• Direction Set (Powell's Method)
• Downhill Simplex (Nelder-Mead)
• Fletcher-Revees conjugate gradient
• Fletcher Xu Hybrid Estimation
• Gauss-Newton
• Genetic Algorithms
• Gradient Descent
• Least-squares
• Levenberg Marquardt
• Liu-Storey conjugate gradient
• Polack-Ribiere conjugate gradient
• Root Finding
• Simulated Annealing

Topic Detection Algorithms

These are algorithms in the Text package that are typically used for topic detection.

• Latent Semantic Analysis (LSA)
• Latent Dirichlet Allocation (LDA)
• Probabilistic Latent Semantic Analysis (pLSA)

Baseline Learners

These are just simple baseline learners that can be compared against.

• Mean
• Most Frequent