Generalized Linear Models

MLTK currently supports the following regularization for generalized linear models (GLMs):

• Ridge
• Lasso
• Elastic Net
• Group Lasso

Response distribution family can be specified by setFamily method for all learners in mltk.predictor.glm package. MLTK supports normal and binomial family so far. Each distribution family will use its default link function.

Ridge

Train a L2-regularized GLM by specifying task.

RidgeLearner learner = new RidgeLearner();
learner.setTask(Task.REGRESSION);
learner.setLambda(0.01);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet);

Train a L2-regularized GLM by specifying family.

RidgeLearner learner = new RidgeLearner();
learner.setLambda(0.01);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet, Family.GAUSSIAN);

Train a L2-regularized GLM by specific method.

RidgeLearner learner = new RidgeLearner();
GLM glm = learner.buildGaussianRegressor(trainSet, 1000, 0.01);

Train L2-regularized GLMs by specifying lambdas.

double[] lambdas = {0.001, 0.01, 0.1};
RidgeLearner learner = new RidgeLearner();
GLM[] glms = learner.buildGaussianRegressors(trainSet, 100, lambdas);

Lasso

Train a L1-regularized GLM by specifying task.

LassoLearner learner = new LassoLearner();
learner.setTask(Task.REGRESSION);
learner.setLambda(0.01);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet);

Train a L1-regularized GLM by specifying family.

LassoLearner learner = new LassoLearner();
learner.setLambda(0.01);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet, Family.GAUSSIAN);

Train a L1-regularized GLM by specific method.

LassoLearner learner = new LassoLearner();
GLM glm = learner.buildGaussianRegressor(trainSet, 1000, 0.01);

Train L1-regularized GLMs by specifying number of lambdas and minimum lambda ratio. This method starts with the largest lambda that sets all weights to zero and gradually decreases the lambda. The minimum lambda is minimum lambda ratio * max lambda.

RidgeLearner learner = new RidgeLearner();
GLM[] glms = learner.buildGaussianRegressors(trainSet, 100, 10, 1e-6);

Elastic Net

Train a elastic-net regularized GLM by specifying task. The regularization parameter on L1 penalty term is l1 ratio * lambda and the regularization parameter on L2 penalty term is (1 - l1 ratio) * lambda.

ElasticNetLearner learner = new ElasticNetLearner();
learner.setTask(Task.REGRESSION);
learner.setLambda(0.01);
learner.setL1Ratio(0.1);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet);

Train a elastic-net regularized GLM by specifying family.

ElasticNetLearner learner = new ElasticNetLearner();
learner.setLambda(0.01);
learner.setL1Ratio(0.1);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet, Family.GAUSSIAN);

Train a elastic-net regularized GLM by specific method.

ElasticNetLearner learner = new ElasticNetLearner();
GLM glm = learner.buildGaussianRegressor(trainSet, 1000, 0.01, 0.1);

Train elastic-net regularized GLMs by specifying number of lambdas and minimum lambda ratio. This method starts with the largest lambda that sets all weights to zero and gradually decreases the lambda. The minimum lambda is minimum lambda ratio * max lambda.

ElasticNetLearner learner = new ElasticNetLearner();
GLM[] glms = learner.buildGaussianRegressors(trainSet, 100, 10, 1e-6, 0.1);

Group Lasso

Assume we have prepared the following groups of features.

List<int[]> groups = new ArrayList<>();
groups.add(new int[] {0, 1, 2});
groups.add(new int[] {3, 4});

Train a group lasso model by specifying task.

GroupLassoLearner learner = new GroupLassoLearner();
learner.setTask(Task.REGRESSION);
learner.setLambda(0.01);
learner.setGroups(groups);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet);

Train a group lasso model by specifying family.

GroupLassoLearner learner = new GroupLassoLearner();
learner.setLambda(0.01);
learner.setGroups(groups);
learner.setMaxNumIters(1000);

GLM glm = learner.build(trainSet, Family.GAUSSIAN);

Train a group lasso model by specific method.

GroupLassoLearner learner = new GroupLassoLearner();
GLM glm = learner.buildGaussianRegressor(trainSet, groups, 1000, 0.01);

Train group lasso models by specifying number of lambdas and minimum lambda ratio. This method starts with the largest lambda that sets all weights to zero and gradually decreases the lambda. The minimum lambda is minimum lambda ratio * max lambda.

GroupLassoLearner learner = new GroupLassoLearner();
GLM[] glms = learner.buildGaussianRegressors(trainSet, groups, 100, 10, 1e-6);