Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

port version 1.91

+ support for linear support vector regression
  • Loading branch information...
commit 283f7f3d4e465cc07138aca536732520e2fe9ddf 1 parent bf0ec6f
Benedikt Waldvogel authored
View
128 README
@@ -29,11 +29,12 @@ The two most important methods that you might be interested in are:
-------------------------------------------------------------------------------
-LIBLINEAR is a simple package for solving large-scale regularized
-linear classification. It currently supports L2-regularized logistic
-regression/L2-loss support vector classification/L1-loss support vector
-classification, and L1-regularized L2-loss support vector classification/
-logistic regression. This document explains the usage of LIBLINEAR.
+LIBLINEAR is a simple package for solving large-scale regularized linear
+classification and regression. It currently supports
+- L2-regularized logistic regression/L2-loss support vector classification/L1-loss support vector classification
+- L1-regularized L2-loss support vector classification/L1-regularized logistic regression
+- L2-regularized L2-loss support vector regression/L1-loss support vector regression.
+This document explains the usage of LIBLINEAR.
To get started, please read the ``Quick Start'' section first.
For developers, please check the ``Library Usage'' section to learn
@@ -59,8 +60,8 @@ When to use LIBLINEAR but not LIBSVM
There are some large data for which with/without nonlinear mappings
gives similar performances. Without using kernels, one can
-efficiently train a much larger set via a linear classifier. These
-data usually have a large number of features. Document classification
+efficiently train a much larger set via linear classification/regression.
+These data usually have a large number of features. Document classification
is an example.
Warning: While generally liblinear is very fast, its default solver
@@ -128,25 +129,34 @@ and mark
Usage: train [options] training_set_file [model_file]
options:
-s type : set type of solver (default 1)
- 0 -- L2-regularized logistic regression (primal)
- 1 -- L2-regularized L2-loss support vector classification (dual)
- 2 -- L2-regularized L2-loss support vector classification (primal)
- 3 -- L2-regularized L1-loss support vector classification (dual)
- 4 -- multi-class support vector classification by Crammer and Singer
- 5 -- L1-regularized L2-loss support vector classification
- 6 -- L1-regularized logistic regression
- 7 -- L2-regularized logistic regression (dual)
+ 0 -- L2-regularized logistic regression (primal)
+ 1 -- L2-regularized L2-loss support vector classification (dual)
+ 2 -- L2-regularized L2-loss support vector classification (primal)
+ 3 -- L2-regularized L1-loss support vector classification (dual)
+ 4 -- multi-class support vector classification by Crammer and Singer
+ 5 -- L1-regularized L2-loss support vector classification
+ 6 -- L1-regularized logistic regression
+ 7 -- L2-regularized logistic regression (dual)
+ 11 -- L2-regularized L2-loss epsilon support vector regression (primal)
+ 12 -- L2-regularized L2-loss epsilon support vector regression (dual)
+ 13 -- L2-regularized L1-loss epsilon support vector regression (dual)
-c cost : set the parameter C (default 1)
+-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-e epsilon : set tolerance of termination criterion
-s 0 and 2
|f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2,
where f is the primal function and pos/neg are # of
positive/negative data (default 0.01)
+ -s 11
+ |f'(w)|_2 <= eps*|f'(w0)|_2 (default 0.001)
-s 1, 3, 4 and 7
Dual maximal violation <= eps; similar to libsvm (default 0.1)
-s 5 and 6
|f'(w)|_inf <= eps*min(pos,neg)/l*|f'(w0)|_inf,
where f is the primal function (default 0.01)
+ -s 12 and 13\n"
+ |f'(alpha)|_1 <= eps |f'(alpha0)|,
+ where f is the dual function (default 0.1)
-B bias : if bias >= 0, instance x becomes [x; bias]; if < 0, no bias term added (default -1)
-wi weight: weights adjust the parameter C of different classes (see README for details)
-v n: n-fold cross validation mode
@@ -183,23 +193,40 @@ For L1-regularized logistic regression (-s 6), we solve
min_w \sum |w_j| + C \sum log(1 + exp(-y_i w^Tx_i))
+For L2-regularized logistic regression (-s 7), we solve
+
+min_alpha 0.5(alpha^T Q alpha) + \sum alpha_i*log(alpha_i) + \sum (C-alpha_i)*log(C-alpha_i) - a constant
+ s.t. 0 <= alpha_i <= C,
+
where
Q is a matrix with Q_ij = y_i y_j x_i^T x_j.
-For L2-regularized logistic regression (-s 7), we solve
+For L2-regularized L2-loss SVR (-s 11), we solve
-min_alpha 0.5(alpha^T Q alpha) + \sum alpha_i*log(alpha_i) + \sum (C-alpha_i)*log(C-alpha_i) - a constant
- s.t. 0 <= alpha_i <= C,
+min_w w^Tw/2 + C \sum max(0, |y_i-w^Tx_i|-epsilon)^2
+
+For L2-regularized L2-loss SVR dual (-s 12), we solve
+
+min_beta 0.5(beta^T (Q + lambda I/2/C) beta) - y^T beta + \sum |beta_i|
+
+For L2-regularized L1-loss SVR dual (-s 13), we solve
+
+min_beta 0.5(beta^T Q beta) - y^T beta + \sum |beta_i|
+ s.t. -C <= beta_i <= C,
+
+where
+
+Q is a matrix with Q_ij = x_i^T x_j.
If bias >= 0, w becomes [w; w_{n+1}] and x becomes [x; bias].
The primal-dual relationship implies that -s 1 and -s 2 give the same
-model, and -s 0 and -s 7 give the same.
+model, -s 0 and -s 7 give the same, and -s 11 and -s 12 give the same.
-We implement 1-vs-the rest multi-class strategy. In training i
-vs. non_i, their C parameters are (weight from -wi)*C and C,
-respectively. If there are only two classes, we train only one
+We implement 1-vs-the rest multi-class strategy for classification.
+In training i vs. non_i, their C parameters are (weight from -wi)*C
+and C, respectively. If there are only two classes, we train only one
model. Thus weight1*C vs. weight2*C is used. See examples below.
We also implement multi-class SVM by Crammer and Singer (-s 4):
@@ -224,7 +251,10 @@ and C^m_i = C if m = y_i,
Usage: predict [options] test_file model_file output_file
options:
--b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0)
+-b probability_estimates: whether to output probability estimates, 0 or 1 (default 0); currently for logistic regression only
+
+Note that -b is only needed in the prediction phase. This is different
+from the setting of LIBSVM.
Examples
========
@@ -267,8 +297,9 @@ Library Usage
- Function: model* train(const struct problem *prob,
const struct parameter *param);
- This function constructs and returns a linear classification model
- according to the given training data and parameters.
+ This function constructs and returns a linear classification
+ or regression model according to the given training data and
+ parameters.
struct problem describes the problem:
@@ -283,10 +314,10 @@ Library Usage
where `l' is the number of training data. If bias >= 0, we assume
that one additional feature is added to the end of each data
instance. `n' is the number of feature (including the bias feature
- if bias >= 0). `y' is an array containing the target values. And
- `x' is an array of pointers,
- each of which points to a sparse representation (array of feature_node) of one
- training vector.
+ if bias >= 0). `y' is an array containing the target values. (integers
+ in classification, real numbers in regression) And `x' is an array
+ of pointers, each of which points to a sparse representation (array
+ of feature_node) of one training vector.
For example, if we have the following training data:
@@ -311,7 +342,8 @@ Library Usage
[ ] -> (2,0.1) (4,1.4) (5,0.5) (6,1) (-1,?)
[ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (6,1) (-1,?)
- struct parameter describes the parameters of a linear classification model:
+ struct parameter describes the parameters of a linear classification
+ or regression model:
struct parameter
{
@@ -323,9 +355,10 @@ Library Usage
int nr_weight;
int *weight_label;
double* weight;
+ double p;
};
- solver_type can be one of L2R_LR, L2R_L2LOSS_SVC_DUAL, L2R_L2LOSS_SVC, L2R_L1LOSS_SVC_DUAL, MCSVM_CS, L1R_L2LOSS_SVC, L1R_LR, L2R_LR_DUAL.
+ solver_type can be one of L2R_LR, L2R_L2LOSS_SVC_DUAL, L2R_L2LOSS_SVC, L2R_L1LOSS_SVC_DUAL, MCSVM_CS, L1R_L2LOSS_SVC, L1R_LR, L2R_LR_DUAL, L2R_L2LOSS_SVR, L2R_L2LOSS_SVR_DUAL, L2R_L1LOSS_SVR_DUAL.
L2R_LR L2-regularized logistic regression (primal)
L2R_L2LOSS_SVC_DUAL L2-regularized L2-loss support vector classification (dual)
@@ -335,8 +368,12 @@ Library Usage
L1R_L2LOSS_SVC L1-regularized L2-loss support vector classification
L1R_LR L1-regularized logistic regression
L2R_LR_DUAL L2-regularized logistic regression (dual)
+ L2R_L2LOSS_SVR L2-regularized L2-loss support vector regression (primal)
+ L2R_L2LOSS_SVR_DUAL L2-regularized L2-loss support vector regression (dual)
+ L2R_L1LOSS_SVR_DUAL L2-regularized L1-loss support vector regression (dual)
C is the cost of constraints violation.
+ p is the sensitiveness of loss of support vector regression.
eps is the stopping criterion.
nr_weight, weight_label, and weight are used to change the penalty
@@ -368,7 +405,8 @@ Library Usage
param describes the parameters used to obtain the model.
- nr_class and nr_feature are the number of classes and features, respectively.
+ nr_class and nr_feature are the number of classes and features,
+ respectively. nr_class = 2 for regression.
The nr_feature*nr_class array w gives feature weights. We use one
against the rest for multi-class classification, so each feature
@@ -386,7 +424,7 @@ Library Usage
The array label stores class labels.
-- Function: void cross_validation(const problem *prob, const parameter *param, int nr_fold, int *target);
+- Function: void cross_validation(const problem *prob, const parameter *param, int nr_fold, double *target);
This function conducts cross validation. Data are separated to
nr_fold folds. Under given parameters, sequentially each fold is
@@ -396,23 +434,25 @@ Library Usage
The format of prob is same as that for train().
-- Function: int predict(const model *model_, const feature_node *x);
+- Function: double predict(const model *model_, const feature_node *x);
- This functions classifies a test vector using the given
- model. The predicted label is returned.
+ For a classification model, the predicted class for x is returned.
+ For a regression model, the function value of x calculated using
+ the model is returned.
-- Function: int predict_values(const struct model *model_,
+- Function: double predict_values(const struct model *model_,
const struct feature_node *x, double* dec_values);
- This function gives nr_w decision values in the array
- dec_values. nr_w is 1 if there are two classes except multi-class
- svm by Crammer and Singer (-s 4), and is the number of classes otherwise.
+ This function gives nr_w decision values in the array dec_values.
+ nr_w=1 if regression is applied or the number of classes is two. An exception is
+ multi-class svm by Crammer and Singer (-s 4), where nr_w = 2 if there are two classes. For all other situations, nr_w is the
+ number of classes.
- We implement one-vs-the rest multi-class strategy (-s 0,1,2,3) and
- multi-class svm by Crammer and Singer (-s 4) for multi-class SVM.
+ We implement one-vs-the rest multi-class strategy (-s 0,1,2,3,5,6,7)
+ and multi-class svm by Crammer and Singer (-s 4) for multi-class SVM.
The class with the highest decision value is returned.
-- Function: int predict_probability(const struct model *model_,
+- Function: double predict_probability(const struct model *model_,
const struct feature_node *x, double* prob_estimates);
This function gives nr_class probability estimates in the array
@@ -428,10 +468,12 @@ Library Usage
- Function: int get_nr_class(const model *model_);
The function gives the number of classes of the model.
+ For a regression model, 2 is returned.
- Function: void get_labels(const model *model_, int* label);
This function outputs the name of labels into an array called label.
+ For a regression model, label is unchanged.
- Function: const char *check_parameter(const struct problem *prob,
const struct parameter *param);
View
2  pom.xml
@@ -4,7 +4,7 @@
<artifactId>liblinear</artifactId>
<packaging>jar</packaging>
<name>liblinear</name>
- <version>1.9-SNAPSHOT</version>
+ <version>1.91-SNAPSHOT</version>
<description>Java port of Liblinear</description>
<url>http://www.bwaldvogel.de/liblinear-java/</url>
View
47 src/main/java/de/bwaldvogel/liblinear/L2R_L2_SvcFunction.java
@@ -2,49 +2,40 @@
class L2R_L2_SvcFunction implements Function {
- private final Problem prob;
- private final double[] C;
- private final int[] I;
- private final double[] z;
+ protected final Problem prob;
+ protected final double[] C;
+ protected final int[] I;
+ protected final double[] z;
- private int sizeI;
+ protected int sizeI;
- public L2R_L2_SvcFunction( Problem prob, double Cp, double Cn ) {
- int i;
+ public L2R_L2_SvcFunction( Problem prob, double[] C ) {
int l = prob.l;
- int[] y = prob.y;
this.prob = prob;
z = new double[l];
- C = new double[l];
I = new int[l];
-
- for (i = 0; i < l; i++) {
- if (y[i] == 1)
- C[i] = Cp;
- else
- C[i] = Cn;
- }
+ this.C = C;
}
public double fun(double[] w) {
int i;
double f = 0;
- int[] y = prob.y;
+ double[] y = prob.y;
int l = prob.l;
int w_size = get_nr_variable();
Xv(w, z);
+
+ for (i = 0; i < w_size; i++)
+ f += w[i] * w[i];
+ f /= 2.0;
for (i = 0; i < l; i++) {
z[i] = y[i] * z[i];
double d = 1 - z[i];
if (d > 0) f += C[i] * d * d;
}
- f = 2 * f;
- for (i = 0; i < w_size; i++)
- f += w[i] * w[i];
- f /= 2.0;
return (f);
}
@@ -54,13 +45,12 @@ public int get_nr_variable() {
}
public void grad(double[] w, double[] g) {
- int i;
- int[] y = prob.y;
+ double[] y = prob.y;
int l = prob.l;
int w_size = get_nr_variable();
sizeI = 0;
- for (i = 0; i < l; i++) {
+ for (int i = 0; i < l; i++) {
if (z[i] < 1) {
z[sizeI] = C[i] * y[i] * (z[i] - 1);
I[sizeI] = i;
@@ -69,15 +59,14 @@ public void grad(double[] w, double[] g) {
}
subXTv(z, g);
- for (i = 0; i < w_size; i++)
+ for (int i = 0; i < w_size; i++)
g[i] = w[i] + 2 * g[i];
}
public void Hv(double[] s, double[] Hs) {
int i;
- int l = prob.l;
int w_size = get_nr_variable();
- double[] wa = new double[l];
+ double[] wa = new double[sizeI];
subXv(s, wa);
for (i = 0; i < sizeI; i++)
@@ -88,7 +77,7 @@ public void Hv(double[] s, double[] Hs) {
Hs[i] = s[i] + 2 * Hs[i];
}
- private void subXTv(double[] v, double[] XTv) {
+ protected void subXTv(double[] v, double[] XTv) {
int i;
int w_size = get_nr_variable();
@@ -112,7 +101,7 @@ private void subXv(double[] v, double[] Xv) {
}
}
- private void Xv(double[] v, double[] Xv) {
+ protected void Xv(double[] v, double[] Xv) {
for (int i = 0; i < prob.l; i++) {
Xv[i] = 0;
View
67 src/main/java/de/bwaldvogel/liblinear/L2R_L2_SvrFunction.java
@@ -0,0 +1,67 @@
+package de.bwaldvogel.liblinear;
+
+/**
+ * @since 1.91
+ */
+public class L2R_L2_SvrFunction extends L2R_L2_SvcFunction {
+
+ private double p;
+
+ public L2R_L2_SvrFunction( Problem prob, double[] C, double p ) {
+ super(prob, C);
+ this.p = p;
+ }
+
+ @Override
+ public double fun(double[] w) {
+ double f = 0;
+ double[] y = prob.y;
+ int l = prob.l;
+ int w_size = get_nr_variable();
+ double d;
+
+ Xv(w, z);
+
+ for (int i = 0; i < w_size; i++)
+ f += w[i] * w[i];
+ f /= 2;
+ for (int i = 0; i < l; i++) {
+ d = z[i] - y[i];
+ if (d < -p)
+ f += C[i] * (d + p) * (d + p);
+ else if (d > p) f += C[i] * (d - p) * (d - p);
+ }
+
+ return f;
+ }
+
+ @Override
+ public void grad(double[] w, double[] g) {
+ double[] y = prob.y;
+ int l = prob.l;
+ int w_size = get_nr_variable();
+
+ sizeI = 0;
+ for (int i = 0; i < l; i++) {
+ double d = z[i] - y[i];
+
+ // generate index set I
+ if (d < -p) {
+ z[sizeI] = C[i] * (d + p);
+ I[sizeI] = i;
+ sizeI++;
+ } else if (d > p) {
+ z[sizeI] = C[i] * (d - p);
+ I[sizeI] = i;
+ sizeI++;
+ }
+
+ }
+ subXTv(z, g);
+
+ for (int i = 0; i < w_size; i++)
+ g[i] = w[i] + 2 * g[i];
+
+ }
+
+}
View
25 src/main/java/de/bwaldvogel/liblinear/L2R_LrFunction.java
@@ -7,23 +7,14 @@
private final double[] D;
private final Problem prob;
- public L2R_LrFunction( Problem prob, double Cp, double Cn ) {
- int i;
+ public L2R_LrFunction( Problem prob, double[] C ) {
int l = prob.l;
- int[] y = prob.y;
this.prob = prob;
z = new double[l];
D = new double[l];
- C = new double[l];
-
- for (i = 0; i < l; i++) {
- if (y[i] == 1)
- C[i] = Cp;
- else
- C[i] = Cn;
- }
+ this.C = C;
}
@@ -56,11 +47,15 @@ private void XTv(double[] v, double[] XTv) {
public double fun(double[] w) {
int i;
double f = 0;
- int[] y = prob.y;
+ double[] y = prob.y;
int l = prob.l;
int w_size = get_nr_variable();
Xv(w, z);
+
+ for (i = 0; i < w_size; i++)
+ f += w[i] * w[i];
+ f /= 2.0;
for (i = 0; i < l; i++) {
double yz = y[i] * z[i];
if (yz >= 0)
@@ -68,17 +63,13 @@ public double fun(double[] w) {
else
f += C[i] * (-yz + Math.log(1 + Math.exp(yz)));
}
- f = 2.0 * f;
- for (i = 0; i < w_size; i++)
- f += w[i] * w[i];
- f /= 2.0;
return (f);
}
public void grad(double[] w, double[] g) {
int i;
- int[] y = prob.y;
+ double[] y = prob.y;
int l = prob.l;
int w_size = get_nr_variable();
View
529 src/main/java/de/bwaldvogel/liblinear/Linear.java
@@ -28,7 +28,7 @@
*
* <p><em>The port was done by Benedikt Waldvogel (mail at bwaldvogel.de)</em></p>
*
- * @version 1.9-SNAPSHOT
+ * @version 1.91-SNAPSHOT
*/
public class Linear {
@@ -48,7 +48,7 @@
/**
* @param target predicted classes
*/
- public static void crossValidation(Problem prob, Parameter param, int nr_fold, int[] target) {
+ public static void crossValidation(Problem prob, Parameter param, int nr_fold, double[] target) {
int i;
int[] fold_start = new int[nr_fold + 1];
int l = prob.l;
@@ -73,7 +73,7 @@ public static void crossValidation(Problem prob, Parameter param, int nr_fold, i
subprob.n = prob.n;
subprob.l = l - (end - begin);
subprob.x = new Feature[subprob.l][];
- subprob.y = new int[subprob.l];
+ subprob.y = new double[subprob.l];
k = 0;
for (j = 0; j < begin; j++) {
@@ -119,7 +119,7 @@ private static GroupClassesReturn groupClasses(Problem prob, int[] perm) {
int i;
for (i = 0; i < l; i++) {
- int this_label = prob.y[i];
+ int this_label = (int)prob.y[i];
int j;
for (j = 0; j < nr_class; j++) {
if (this_label == label[j]) {
@@ -314,7 +314,7 @@ static void closeQuietly(Closeable c) {
} catch (Throwable t) {}
}
- public static int predict(Model model, Feature[] x) {
+ public static double predict(Model model, Feature[] x) {
double[] dec_values = new double[model.nr_class];
return predictValues(model, x, dec_values);
}
@@ -322,7 +322,7 @@ public static int predict(Model model, Feature[] x) {
/**
* @throws IllegalArgumentException if model is not probabilistic (see {@link Model#isProbabilityModel()})
*/
- public static int predictProbability(Model model, Feature[] x, double[] prob_estimates) throws IllegalArgumentException {
+ public static double predictProbability(Model model, Feature[] x, double[] prob_estimates) throws IllegalArgumentException {
if (!model.isProbabilityModel()) {
throw new IllegalArgumentException("probability output is only supported for logistic regression");
}
@@ -333,7 +333,7 @@ public static int predictProbability(Model model, Feature[] x, double[] prob_est
else
nr_w = nr_class;
- int label = predictValues(model, x, prob_estimates);
+ double label = predictValues(model, x, prob_estimates);
for (int i = 0; i < nr_w; i++)
prob_estimates[i] = 1 / (1 + Math.exp(-prob_estimates[i]));
@@ -351,7 +351,7 @@ public static int predictProbability(Model model, Feature[] x, double[] prob_est
return label;
}
- public static int predictValues(Model model, Feature[] x, double[] dec_values) {
+ public static double predictValues(Model model, Feature[] x, double[] dec_values) {
int n;
if (model.bias >= 0)
n = model.nr_feature + 1;
@@ -379,9 +379,12 @@ public static int predictValues(Model model, Feature[] x, double[] dec_values) {
}
}
- if (model.nr_class == 2)
- return (dec_values[0] > 0) ? model.label[0] : model.label[1];
- else {
+ if (model.nr_class == 2) {
+ if (model.solverType.isSupportVectorRegression())
+ return dec_values[0];
+ else
+ return (dec_values[0] > 0) ? model.label[0] : model.label[1];
+ } else {
int dec_max_idx = 0;
for (int i = 1; i < model.nr_class; i++) {
if (dec_values[i] > dec_values[dec_max_idx]) dec_max_idx = i;
@@ -390,7 +393,6 @@ public static int predictValues(Model model, Feature[] x, double[] dec_values) {
}
}
-
static void printf(Formatter formatter, String format, Object... args) throws IOException {
formatter.format(format, args);
IOException ioException = formatter.ioException();
@@ -416,11 +418,13 @@ public static void saveModel(Writer modelOutput, Model model) throws IOException
printf(formatter, "solver_type %s\n", model.solverType.name());
printf(formatter, "nr_class %d\n", model.nr_class);
- printf(formatter, "label");
- for (int i = 0; i < model.nr_class; i++) {
- printf(formatter, " %d", model.label[i]);
+ if (model.label != null) {
+ printf(formatter, "label");
+ for (int i = 0; i < model.nr_class; i++) {
+ printf(formatter, " %d", model.label[i]);
+ }
+ printf(formatter, "\n");
}
- printf(formatter, "\n");
printf(formatter, "nr_feature %d\n", nr_feature);
printf(formatter, "bias %.16g\n", model.bias);
@@ -471,7 +475,7 @@ private static int GETI(byte[] y, int i) {
* L1-loss and L2-loss SVM dual problems
*<pre>
* min_\alpha 0.5(\alpha^T (Q + D)\alpha) - e^T \alpha,
- * s.t. 0 <= alpha_i <= upper_bound_i,
+ * s.t. 0 <= \alpha_i <= upper_bound_i,
*
* where Qij = yi yj xi^T xj and
* D is a diagonal matrix
@@ -522,19 +526,28 @@ private static void solve_l2r_l1l2_svc(Problem prob, double[] w, double eps, dou
upper_bound[2] = Cp;
}
- for (i = 0; i < w_size; i++)
- w[i] = 0;
for (i = 0; i < l; i++) {
- alpha[i] = 0;
if (prob.y[i] > 0) {
y[i] = +1;
} else {
y[i] = -1;
}
+ }
+
+ // Initial alpha can be set here. Note that
+ // 0 <= alpha[i] <= upper_bound[GETI(i)]
+ for (i = 0; i < l; i++)
+ alpha[i] = 0;
+
+ for (i = 0; i < w_size; i++)
+ w[i] = 0;
+ for (i = 0; i < l; i++) {
QD[i] = diag[GETI(y, i)];
for (Feature xi : prob.x[i]) {
- QD[i] += xi.getValue() * xi.getValue();
+ double val = xi.getValue();
+ QD[i] += val * val;
+ w[xi.getIndex() - 1] += y[i] * alpha[i] * val;
}
index[i] = i;
}
@@ -635,12 +648,206 @@ private static void solve_l2r_l1l2_svc(Problem prob, double[] w, double eps, dou
info("nSV = %d" + NL, nSV);
}
+ // To support weights for instances, use GETI(i) (i)
+ private static int GETI_SVR(int i) {
+ return 0;
+ }
+
+ /**
+ * A coordinate descent algorithm for
+ * L1-loss and L2-loss epsilon-SVR dual problem
+ *
+ * min_\beta 0.5\beta^T (Q + diag(lambda)) \beta - p \sum_{i=1}^l|\beta_i| + \sum_{i=1}^l yi\beta_i,
+ * s.t. -upper_bound_i <= \beta_i <= upper_bound_i,
+ *
+ * where Qij = xi^T xj and
+ * D is a diagonal matrix
+ *
+ * In L1-SVM case:
+ * upper_bound_i = C
+ * lambda_i = 0
+ * In L2-SVM case:
+ * upper_bound_i = INF
+ * lambda_i = 1/(2*C)
+ *
+ * Given:
+ * x, y, p, C
+ * eps is the stopping tolerance
+ *
+ * solution will be put in w
+ *
+ * See Algorithm 4 of Ho and Lin, 2012
+ */
+ private static void solve_l2r_l1l2_svr(Problem prob, double[] w, Parameter param) {
+ int l = prob.l;
+ double C = param.C;
+ double p = param.p;
+ int w_size = prob.n;
+ double eps = param.eps;
+ int i, s, iter = 0;
+ int max_iter = 1000;
+ int active_size = l;
+ int[] index = new int[l];
+
+ double d, G, H;
+ double Gmax_old = Double.POSITIVE_INFINITY;
+ double Gmax_new, Gnorm1_new;
+ double Gnorm1_init = 0; // initialize to 0 to get rid of Eclipse warning/error
+ double[] beta = new double[l];
+ double[] QD = new double[l];
+ double[] y = prob.y;
+
+ // L2R_L2LOSS_SVR_DUAL
+ double[] lambda = new double[] {0.5 / C};
+ double[] upper_bound = new double[] {Double.POSITIVE_INFINITY};
+
+ if (param.solverType == SolverType.L2R_L1LOSS_SVR_DUAL) {
+ lambda[0] = 0;
+ upper_bound[0] = C;
+ }
+
+ // Initial beta can be set here. Note that
+ // -upper_bound <= beta[i] <= upper_bound
+ for (i = 0; i < l; i++)
+ beta[i] = 0;
+
+ for (i = 0; i < w_size; i++)
+ w[i] = 0;
+ for (i = 0; i < l; i++) {
+ QD[i] = 0;
+ for (Feature xi : prob.x[i]) {
+ double val = xi.getValue();
+ QD[i] += val * val;
+ w[xi.getIndex() - 1] += beta[i] * val;
+ }
+
+ index[i] = i;
+ }
+
+
+ while (iter < max_iter) {
+ Gmax_new = 0;
+ Gnorm1_new = 0;
+
+ for (i = 0; i < active_size; i++) {
+ int j = i + random.nextInt(active_size - i);
+ swap(index, i, j);
+ }
+
+ for (s = 0; s < active_size; s++) {
+ i = index[s];
+ G = -y[i] + lambda[GETI_SVR(i)] * beta[i];
+ H = QD[i] + lambda[GETI_SVR(i)];
+
+ for (Feature xi : prob.x[i]) {
+ int ind = xi.getIndex() - 1;
+ double val = xi.getValue();
+ G += val * w[ind];
+ }
+
+ double Gp = G + p;
+ double Gn = G - p;
+ double violation = 0;
+ if (beta[i] == 0) {
+ if (Gp < 0)
+ violation = -Gp;
+ else if (Gn > 0)
+ violation = Gn;
+ else if (Gp > Gmax_old && Gn < -Gmax_old) {
+ active_size--;
+ swap(index, s, active_size);
+ s--;
+ continue;
+ }
+ } else if (beta[i] >= upper_bound[GETI_SVR(i)]) {
+ if (Gp > 0)
+ violation = Gp;
+ else if (Gp < -Gmax_old) {
+ active_size--;
+ swap(index, s, active_size);
+ s--;
+ continue;
+ }
+ } else if (beta[i] <= -upper_bound[GETI_SVR(i)]) {
+ if (Gn < 0)
+ violation = -Gn;
+ else if (Gn > Gmax_old) {
+ active_size--;
+ swap(index, s, active_size);
+ s--;
+ continue;
+ }
+ } else if (beta[i] > 0)
+ violation = Math.abs(Gp);
+ else
+ violation = Math.abs(Gn);
+
+ Gmax_new = Math.max(Gmax_new, violation);
+ Gnorm1_new += violation;
+
+ // obtain Newton direction d
+ if (Gp < H * beta[i])
+ d = -Gp / H;
+ else if (Gn > H * beta[i])
+ d = -Gn / H;
+ else
+ d = -beta[i];
+
+ if (Math.abs(d) < 1.0e-12) continue;
+
+ double beta_old = beta[i];
+ beta[i] = Math.min(Math.max(beta[i] + d, -upper_bound[GETI_SVR(i)]), upper_bound[GETI_SVR(i)]);
+ d = beta[i] - beta_old;
+
+ if (d != 0) {
+ for (Feature xi : prob.x[i]) {
+ w[xi.getIndex() - 1] += d * xi.getValue();
+ }
+ }
+ }
+
+ if (iter == 0) Gnorm1_init = Gnorm1_new;
+ iter++;
+ if (iter % 10 == 0) info(".");
+
+ if (Gnorm1_new <= eps * Gnorm1_init) {
+ if (active_size == l)
+ break;
+ else {
+ active_size = l;
+ info("*");
+ Gmax_old = Double.POSITIVE_INFINITY;
+ continue;
+ }
+ }
+
+ Gmax_old = Gmax_new;
+ }
+
+ info("%noptimization finished, #iter = %d%n", iter);
+ if (iter >= max_iter) info("%nWARNING: reaching max number of iterations%nUsing -s 11 may be faster%n%n");
+
+ // calculate objective value
+ double v = 0;
+ int nSV = 0;
+ for (i = 0; i < w_size; i++)
+ v += w[i] * w[i];
+ v = 0.5 * v;
+ for (i = 0; i < l; i++) {
+ v += p * Math.abs(beta[i]) - y[i] * beta[i] + 0.5 * lambda[GETI_SVR(i)] * beta[i] * beta[i];
+ if (beta[i] != 0) nSV++;
+ }
+
+ info("Objective value = %f%n", v);
+ info("nSV = %d%n", nSV);
+ }
+
/**
* A coordinate descent algorithm for
* the dual of L2-regularized logistic regression problems
*<pre>
- * min_\alpha 0.5(\alpha^T Q \alpha) + \sum \alpha_i log (\alpha_i) + (upper_bound_i - alpha_i) log (upper_bound_i - alpha_i) ,
- * s.t. 0 <= alpha_i <= upper_bound_i,
+ * min_\alpha 0.5(\alpha^T Q \alpha) + \sum \alpha_i log (\alpha_i) + (upper_bound_i - \alpha_i) log (upper_bound_i - \alpha_i) ,
+ * s.t. 0 <= \alpha_i <= upper_bound_i,
*
* where Qij = yi yj xi^T xj and
* upper_bound_i = Cp if y_i = 1
@@ -671,21 +878,30 @@ private static void solve_l2r_lr_dual(Problem prob, double w[], double eps, doub
double innereps_min = Math.min(1e-8, eps);
double upper_bound[] = new double[] {Cn, 0, Cp};
- for (i = 0; i < w_size; i++)
- w[i] = 0;
for (i = 0; i < l; i++) {
if (prob.y[i] > 0) {
y[i] = +1;
} else {
y[i] = -1;
}
+ }
+
+ // Initial alpha can be set here. Note that
+ // 0 < alpha[i] < upper_bound[GETI(i)]
+ // alpha[2*i] + alpha[2*i+1] = upper_bound[GETI(i)]
+ for (i = 0; i < l; i++) {
alpha[2 * i] = Math.min(0.001 * upper_bound[GETI(y, i)], 1e-8);
alpha[2 * i + 1] = upper_bound[GETI(y, i)] - alpha[2 * i];
+ }
+ for (i = 0; i < w_size; i++)
+ w[i] = 0;
+ for (i = 0; i < l; i++) {
xTx[i] = 0;
for (Feature xi : prob.x[i]) {
- xTx[i] += (xi.getValue()) * (xi.getValue());
- w[xi.getIndex() - 1] += y[i] * alpha[2 * i] * xi.getValue();
+ double val = xi.getValue();
+ xTx[i] += val * val;
+ w[xi.getIndex() - 1] += y[i] * alpha[2 * i] * val;
}
index[i] = i;
}
@@ -819,6 +1035,10 @@ private static void solve_l1r_l2_svc(Problem prob_col, double[] w, double eps, d
double[] C = new double[] {Cn, 0, Cp};
+ // Initial w can be set here.
+ for (j = 0; j < w_size; j++)
+ w[j] = 0;
+
for (j = 0; j < l; j++) {
b[j] = 1;
if (prob_col.y[j] > 0)
@@ -827,13 +1047,14 @@ private static void solve_l1r_l2_svc(Problem prob_col, double[] w, double eps, d
y[j] = -1;
}
for (j = 0; j < w_size; j++) {
- w[j] = 0;
index[j] = j;
xj_sq[j] = 0;
for (Feature xi : prob_col.x[j]) {
int ind = xi.getIndex() - 1;
- double val = xi.getValue();
xi.setValue(xi.getValue() * y[ind]); // x->value stores yi*xij
+ double val = xi.getValue();
+ b[ind] -= w[j] * val;
+
xj_sq[j] += C[GETI(y, ind)] * val * val;
}
}
@@ -890,9 +1111,9 @@ else if (Gp > Gmax_old / l && Gn < -Gmax_old / l) {
Gnorm1_new += violation;
// obtain Newton direction d
- if (Gp <= H * w[j])
+ if (Gp < H * w[j])
d = -Gp / H;
- else if (Gn >= H * w[j])
+ else if (Gn > H * w[j])
d = -Gn / H;
else
d = -w[j];
@@ -1041,7 +1262,7 @@ private static void solve_l1r_lr(Problem prob_col, double[] w, double eps, doubl
double nu = 1e-12;
double inner_eps = 1;
double sigma = 0.01;
- double w_norm = 0, w_norm_new;
+ double w_norm, w_norm_new;
double z, G, H;
double Gnorm1_init = 0; // eclipse moans this variable might not be initialized
double Gmax_old = Double.POSITIVE_INFINITY;
@@ -1064,27 +1285,40 @@ private static void solve_l1r_lr(Problem prob_col, double[] w, double eps, doubl
double[] C = {Cn, 0, Cp};
+ // Initial w can be set here.
+ for (j = 0; j < w_size; j++)
+ w[j] = 0;
+
for (j = 0; j < l; j++) {
if (prob_col.y[j] > 0)
y[j] = 1;
else
y[j] = -1;
- // assume initial w is 0
- exp_wTx[j] = 1;
- tau[j] = C[GETI(y, j)] * 0.5;
- D[j] = C[GETI(y, j)] * 0.25;
+ exp_wTx[j] = 0;
}
+
+ w_norm = 0;
for (j = 0; j < w_size; j++) {
- w[j] = 0;
+ w_norm += Math.abs(w[j]);
wpd[j] = w[j];
index[j] = j;
xjneg_sum[j] = 0;
for (Feature x : prob_col.x[j]) {
int ind = x.getIndex() - 1;
- if (y[ind] == -1) xjneg_sum[j] += C[GETI(y, ind)] * x.getValue();
+ double val = x.getValue();
+ exp_wTx[ind] += w[j] * val;
+ if (y[ind] == -1) {
+ xjneg_sum[j] += C[GETI(y, ind)] * val;
+ }
}
}
+ for (j = 0; j < l; j++) {
+ exp_wTx[j] = Math.exp(exp_wTx[j]);
+ double tau_tmp = 1 / (1 + exp_wTx[j]);
+ tau[j] = C[GETI(y, j)] * tau_tmp;
+ D[j] = C[GETI(y, j)] * exp_wTx[j] * tau_tmp * tau_tmp;
+ }
while (newton_iter < max_newton_iter) {
Gmax_new = 0;
@@ -1183,9 +1417,9 @@ else if (Gp > QP_Gmax_old / l && Gn < -QP_Gmax_old / l) {
QP_Gnorm1_new += violation;
// obtain solution of one-variable problem
- if (Gp <= H * wpd[j])
+ if (Gp < H * wpd[j])
z = -Gp / H;
- else if (Gn >= H * wpd[j])
+ else if (Gn > H * wpd[j])
z = -Gn / H;
else
z = -wpd[j];
@@ -1218,7 +1452,7 @@ else if (Gn >= H * wpd[j])
QP_Gmax_old = QP_Gmax_new;
}
- if (iter >= max_iter) info("WARNING: reaching max number of inner iterations\n");
+ if (iter >= max_iter) info("WARNING: reaching max number of inner iterations%n");
delta = 0;
w_norm_new = 0;
@@ -1321,7 +1555,7 @@ static Problem transpose(Problem prob) {
Problem prob_col = new Problem();
prob_col.l = l;
prob_col.n = n;
- prob_col.y = new int[l];
+ prob_col.y = new double[l];
prob_col.x = new Feature[n][];
for (int i = 0; i < l; i++)
@@ -1368,6 +1602,7 @@ static void swap(IntArrayPointer array, int idxA, int idxB) {
array.set(idxB, temp);
}
+
/**
* @throws IllegalArgumentException if the feature nodes of prob are not sorted in ascending order
*/
@@ -1395,123 +1630,158 @@ public static Model train(Problem prob, Parameter param) {
model.nr_feature = n - 1;
else
model.nr_feature = n;
+
model.solverType = param.solverType;
model.bias = prob.bias;
- int[] perm = new int[l];
- // group training data of the same class
- GroupClassesReturn rv = groupClasses(prob, perm);
- int nr_class = rv.nr_class;
- int[] label = rv.label;
- int[] start = rv.start;
- int[] count = rv.count;
-
- model.nr_class = nr_class;
- model.label = new int[nr_class];
- for (int i = 0; i < nr_class; i++)
- model.label[i] = label[i];
-
- // calculate weighted C
- double[] weighted_C = new double[nr_class];
- for (int i = 0; i < nr_class; i++) {
- weighted_C[i] = param.C;
- }
+ if (param.solverType == SolverType.L2R_L2LOSS_SVR || //
+ param.solverType == SolverType.L2R_L1LOSS_SVR_DUAL || //
+ param.solverType == SolverType.L2R_L2LOSS_SVR_DUAL) {
+ model.w = new double[w_size];
+ model.nr_class = 2;
+ model.label = null;
- for (int i = 0; i < param.getNumWeights(); i++) {
- int j;
- for (j = 0; j < nr_class; j++)
- if (param.weightLabel[i] == label[j]) break;
- if (j == nr_class) throw new IllegalArgumentException("class label " + param.weightLabel[i] + " specified in weight is not found");
-
- weighted_C[j] *= param.weight[i];
- }
+ checkProblemSize(n, model.nr_class);
- // constructing the subproblem
- Feature[][] x = new Feature[l][];
- for (int i = 0; i < l; i++)
- x[i] = prob.x[perm[i]];
+ train_one(prob, param, model.w, 0, 0);
+ } else {
+ int[] perm = new int[l];
- Problem sub_prob = new Problem();
- sub_prob.l = l;
- sub_prob.n = n;
- sub_prob.x = new Feature[sub_prob.l][];
- sub_prob.y = new int[sub_prob.l];
+ // group training data of the same class
+ GroupClassesReturn rv = groupClasses(prob, perm);
+ int nr_class = rv.nr_class;
+ int[] label = rv.label;
+ int[] start = rv.start;
+ int[] count = rv.count;
- for (int k = 0; k < sub_prob.l; k++)
- sub_prob.x[k] = x[k];
+ checkProblemSize(n, nr_class);
- // verify the size and throw an exception early if the problem is too large
- if (n >= Integer.MAX_VALUE / nr_class || n * nr_class < 0) {
- throw new IllegalArgumentException("'number of classes' * 'number of instances' is too large: " + nr_class + "*" + n);
- }
+ model.nr_class = nr_class;
+ model.label = new int[nr_class];
+ for (int i = 0; i < nr_class; i++)
+ model.label[i] = label[i];
- // multi-class svm by Crammer and Singer
- if (param.solverType == SolverType.MCSVM_CS) {
- model.w = new double[n * nr_class];
- for (int i = 0; i < nr_class; i++) {
- for (int j = start[i]; j < start[i] + count[i]; j++) {
- sub_prob.y[j] = i;
- }
+ // calculate weighted C
+ double[] weighted_C = new double[nr_class];
+ for (int i = 0; i < nr_class; i++)
+ weighted_C[i] = param.C;
+ for (int i = 0; i < param.getNumWeights(); i++) {
+ int j;
+ for (j = 0; j < nr_class; j++)
+ if (param.weightLabel[i] == label[j]) break;
+
+ if (j == nr_class) throw new IllegalArgumentException("class label " + param.weightLabel[i] + " specified in weight is not found");
+ weighted_C[j] *= param.weight[i];
}
- SolverMCSVM_CS solver = new SolverMCSVM_CS(sub_prob, nr_class, weighted_C, param.eps);
- solver.solve(model.w);
- } else {
- if (nr_class == 2) {
- model.w = new double[w_size];
+ // constructing the subproblem
+ Feature[][] x = new Feature[l][];
+ for (int i = 0; i < l; i++)
+ x[i] = prob.x[perm[i]];
- int e0 = start[0] + count[0];
- int k = 0;
- for (; k < e0; k++)
- sub_prob.y[k] = +1;
- for (; k < sub_prob.l; k++)
- sub_prob.y[k] = -1;
+ Problem sub_prob = new Problem();
+ sub_prob.l = l;
+ sub_prob.n = n;
+ sub_prob.x = new Feature[sub_prob.l][];
+ sub_prob.y = new double[sub_prob.l];
- train_one(sub_prob, param, model.w, weighted_C[0], weighted_C[1]);
- } else {
- model.w = new double[w_size * nr_class];
- double[] w = new double[w_size];
+ for (int k = 0; k < sub_prob.l; k++)
+ sub_prob.x[k] = x[k];
+
+ // multi-class svm by Crammer and Singer
+ if (param.solverType == SolverType.MCSVM_CS) {
+ model.w = new double[n * nr_class];
for (int i = 0; i < nr_class; i++) {
- int si = start[i];
- int ei = si + count[i];
+ for (int j = start[i]; j < start[i] + count[i]; j++) {
+ sub_prob.y[j] = i;
+ }
+ }
+ SolverMCSVM_CS solver = new SolverMCSVM_CS(sub_prob, nr_class, weighted_C, param.eps);
+ solver.solve(model.w);
+ } else {
+ if (nr_class == 2) {
+ model.w = new double[w_size];
+
+ int e0 = start[0] + count[0];
int k = 0;
- for (; k < si; k++)
- sub_prob.y[k] = -1;
- for (; k < ei; k++)
+ for (; k < e0; k++)
sub_prob.y[k] = +1;
for (; k < sub_prob.l; k++)
sub_prob.y[k] = -1;
- train_one(sub_prob, param, w, weighted_C[i], param.C);
-
- for (int j = 0; j < n; j++)
- model.w[j * nr_class + i] = w[j];
+ train_one(sub_prob, param, model.w, weighted_C[0], weighted_C[1]);
+ } else {
+ model.w = new double[w_size * nr_class];
+ double[] w = new double[w_size];
+ for (int i = 0; i < nr_class; i++) {
+ int si = start[i];
+ int ei = si + count[i];
+
+ int k = 0;
+ for (; k < si; k++)
+ sub_prob.y[k] = -1;
+ for (; k < ei; k++)
+ sub_prob.y[k] = +1;
+ for (; k < sub_prob.l; k++)
+ sub_prob.y[k] = -1;
+
+ train_one(sub_prob, param, w, weighted_C[i], param.C);
+
+ for (int j = 0; j < n; j++)
+ model.w[j * nr_class + i] = w[j];
+ }
}
}
-
}
return model;
}
+ /**
+ * verify the size and throw an exception early if the problem is too large
+ */
+ private static void checkProblemSize(int n, int nr_class) {
+ if (n >= Integer.MAX_VALUE / nr_class || n * nr_class < 0) {
+ throw new IllegalArgumentException("'number of classes' * 'number of instances' is too large: " + nr_class + "*" + n);
+ }
+ }
+
private static void train_one(Problem prob, Parameter param, double[] w, double Cp, double Cn) {
double eps = param.eps;
int pos = 0;
for (int i = 0; i < prob.l; i++)
- if (prob.y[i] == +1) pos++;
+ if (prob.y[i] > 0) {
+ pos++;
+ }
int neg = prob.l - pos;
+ double primal_solver_tol = eps * Math.max(Math.min(pos, neg), 1) / prob.l;
+
Function fun_obj = null;
switch (param.solverType) {
case L2R_LR: {
- fun_obj = new L2R_LrFunction(prob, Cp, Cn);
- Tron tron_obj = new Tron(fun_obj, eps * Math.min(pos, neg) / prob.l);
+ double[] C = new double[prob.l];
+ for (int i = 0; i < prob.l; i++) {
+ if (prob.y[i] > 0)
+ C[i] = Cp;
+ else
+ C[i] = Cn;
+ }
+ fun_obj = new L2R_LrFunction(prob, C);
+ Tron tron_obj = new Tron(fun_obj, primal_solver_tol);
tron_obj.tron(w);
break;
}
case L2R_L2LOSS_SVC: {
- fun_obj = new L2R_L2_SvcFunction(prob, Cp, Cn);
- Tron tron_obj = new Tron(fun_obj, eps * Math.min(pos, neg) / prob.l);
+ double[] C = new double[prob.l];
+ for (int i = 0; i < prob.l; i++) {
+ if (prob.y[i] > 0)
+ C[i] = Cp;
+ else
+ C[i] = Cn;
+ }
+ fun_obj = new L2R_L2_SvcFunction(prob, C);
+ Tron tron_obj = new Tron(fun_obj, primal_solver_tol);
tron_obj.tron(w);
break;
}
@@ -1523,17 +1793,32 @@ private static void train_one(Problem prob, Parameter param, double[] w, double
break;
case L1R_L2LOSS_SVC: {
Problem prob_col = transpose(prob);
- solve_l1r_l2_svc(prob_col, w, eps * Math.min(pos, neg) / prob.l, Cp, Cn);
+ solve_l1r_l2_svc(prob_col, w, primal_solver_tol, Cp, Cn);
break;
}
case L1R_LR: {
Problem prob_col = transpose(prob);
- solve_l1r_lr(prob_col, w, eps * Math.min(pos, neg) / prob.l, Cp, Cn);
+ solve_l1r_lr(prob_col, w, primal_solver_tol, Cp, Cn);
break;
}
case L2R_LR_DUAL:
solve_l2r_lr_dual(prob, w, eps, Cp, Cn);
break;
+ case L2R_L2LOSS_SVR: {
+ double[] C = new double[prob.l];
+ for (int i = 0; i < prob.l; i++)
+ C[i] = param.C;
+
+ fun_obj = new L2R_L2_SvrFunction(prob, C, param.p);
+ Tron tron_obj = new Tron(fun_obj, param.eps);
+ tron_obj.tron(w);
+ break;
+ }
+ case L2R_L1LOSS_SVR_DUAL:
+ case L2R_L2LOSS_SVR_DUAL:
+ solve_l2r_l1l2_svr(prob, w, param);
+ break;
+
default:
throw new IllegalStateException("unknown solver type: " + param.solverType);
}
View
22 src/main/java/de/bwaldvogel/liblinear/Parameter.java
@@ -16,10 +16,17 @@
int[] weightLabel = null;
- public Parameter( SolverType solverType, double C, double eps ) {
+ double p;
+
+ public Parameter( SolverType solver, double C, double eps ) {
+ this(solver, C, eps, 0.1);
+ }
+
+ public Parameter( SolverType solverType, double C, double eps, double p ) {
setSolverType(solverType);
setC(C);
setEps(eps);
+ setP(p);
}
/**
@@ -97,4 +104,17 @@ public void setSolverType(SolverType solverType) {
public SolverType getSolverType() {
return solverType;
}
+
+
+ /**
+ * set the epsilon in loss function of epsilon-SVR (default 0.1)
+ */
+ public void setP(double p) {
+ if (p < 0) throw new IllegalArgumentException("p must not be less than 0");
+ this.p = p;
+ }
+
+ public double getP() {
+ return p;
+ }
}
View
31 src/main/java/de/bwaldvogel/liblinear/Predict.java
@@ -34,6 +34,8 @@
static void doPredict(BufferedReader reader, Writer writer, Model model) throws IOException {
int correct = 0;
int total = 0;
+ double error = 0;
+ double sump = 0, sumt = 0, sumpp = 0, sumtt = 0, sumpt = 0;
int nr_class = model.getNrClass();
double[] prob_estimates = null;
@@ -65,10 +67,10 @@ static void doPredict(BufferedReader reader, Writer writer, Model model) throws
while ((line = reader.readLine()) != null) {
List<Feature> x = new ArrayList<Feature>();
StringTokenizer st = new StringTokenizer(line, " \t\n");
- int target_label;
+ double target_label;
try {
String label = st.nextToken();
- target_label = atoi(label);
+ target_label = atof(label);
} catch (NoSuchElementException e) {
throw new RuntimeException("Wrong input format at line " + (total + 1), e);
}
@@ -101,31 +103,46 @@ static void doPredict(BufferedReader reader, Writer writer, Model model) throws
Feature[] nodes = new Feature[x.size()];
nodes = x.toArray(nodes);
- int predict_label;
+ double predict_label;
if (flag_predict_probability) {
assert prob_estimates != null;
predict_label = Linear.predictProbability(model, nodes, prob_estimates);
- printf(out, "%d", predict_label);
+ printf(out, "%g", predict_label);
for (int j = 0; j < model.nr_class; j++)
printf(out, " %g", prob_estimates[j]);
printf(out, "\n");
} else {
predict_label = Linear.predict(model, nodes);
- printf(out, "%d\n", predict_label);
+ printf(out, "%g\n", predict_label);
}
if (predict_label == target_label) {
++correct;
}
+
+ error += (predict_label - target_label) * (predict_label - target_label);
+ sump += predict_label;
+ sumt += target_label;
+ sumpp += predict_label * predict_label;
+ sumtt += target_label * target_label;
+ sumpt += predict_label * target_label;
++total;
}
- System.out.printf("Accuracy = %g%% (%d/%d)%n", (double)correct / total * 100, correct, total);
+
+ if (model.solverType.isSupportVectorRegression()) //
+ {
+ System.out.printf("Mean squared error = %g (regression)%n", error / total);
+ System.out.printf("Squared correlation coefficient = %g (regression)%n", //
+ ((total * sumpt - sump * sumt) * (total * sumpt - sump * sumt)) / ((total * sumpp - sump * sump) * (total * sumtt - sumt * sumt)));
+ } else {
+ System.out.printf("Accuracy = %g%% (%d/%d)%n", (double)correct / total * 100, correct, total);
+ }
}
private static void exit_with_help() {
System.out.printf("Usage: predict [options] test_file model_file output_file%n" + "options:%n"
- + "-b probability_estimates: whether to output probability estimates, 0 or 1 (default 0)%n");
+ + "-b probability_estimates: whether to output probability estimates, 0 or 1 (default 0); currently for logistic regression only%n");
System.exit(1);
}
View
2  src/main/java/de/bwaldvogel/liblinear/Problem.java
@@ -40,7 +40,7 @@
public int n;
/** an array containing the target values */
- public int[] y;
+ public double[] y;
/** array of sparse feature nodes */
public Feature[][] x;
View
25 src/main/java/de/bwaldvogel/liblinear/SolverMCSVM_CS.java
@@ -61,7 +61,7 @@ public SolverMCSVM_CS( Problem prob, int nr_class, double[] weighted_C, double e
}
private int GETI(int i) {
- return prob.y[i];
+ return (int)prob.y[i];
}
private boolean be_shrunk(int i, int m, int yi, double alpha_i, double minG) {
@@ -86,9 +86,15 @@ public void solve(double[] w) {
int[] active_size_i = new int[l];
double eps_shrink = Math.max(10.0 * eps, 1.0); // stopping tolerance for shrinking
boolean start_from_all = true;
- // initial
+
+ // Initial alpha can be set here. Note that
+ // sum_m alpha[i*nr_class+m] = 0, for all i=1,...,l-1
+ // alpha[i*nr_class+m] <= C[GETI(i)] if prob->y[i] == m
+ // alpha[i*nr_class+m] <= 0 if prob->y[i] != m
+ // If initial alpha isn't zero, uncomment the for loop below to initialize w
for (i = 0; i < l * nr_class; i++)
alpha[i] = 0;
+
for (i = 0; i < w_size * nr_class; i++)
w[i] = 0;
for (i = 0; i < l; i++) {
@@ -96,10 +102,15 @@ public void solve(double[] w) {
alpha_index[i * nr_class + m] = m;
QD[i] = 0;
for (Feature xi : prob.x[i]) {
- QD[i] += xi.getValue() * xi.getValue();
+ double val = xi.getValue();
+ QD[i] += val * val;
+
+ // Uncomment the for loop if initial alpha isn't zero
+ // for(m=0; m<nr_class; m++)
+ // w[(xi->index-1)*nr_class+m] += alpha[i*nr_class+m]*val;
}
active_size_i[i] = nr_class;
- y_index[i] = prob.y[i];
+ y_index[i] = (int)prob.y[i];
index[i] = i;
}
@@ -145,7 +156,7 @@ public void solve(double[] w) {
if (G[m] > maxG) maxG = G[m];
}
if (y_index[i] < active_size_i[i]) {
- if (alpha_i.get(prob.y[i]) < C[GETI(i)] && G[y_index[i]] < minG) {
+ if (alpha_i.get((int)prob.y[i]) < C[GETI(i)] && G[y_index[i]] < minG) {
minG = G[y_index[i]];
}
}
@@ -239,7 +250,7 @@ public void solve(double[] w) {
if (Math.abs(alpha[i]) > 0) nSV++;
}
for (i = 0; i < l; i++)
- v -= alpha[i * nr_class + prob.y[i]];
+ v -= alpha[i * nr_class + (int)prob.y[i]];
info("Objective value = %f%n", v);
info("nSV = %d%n", nSV);
@@ -260,8 +271,8 @@ private void solve_sub_problem(double A_i, int yi, double C_yi, int active_i, do
double beta = D[0] - A_i * C_yi;
for (r = 1; r < active_i && beta < r * D[r]; r++)
beta += D[r];
-
beta /= r;
+
for (r = 0; r < active_i; r++) {
if (r == yi)
alpha_new[r] = Math.min(C_yi, (beta - B[r]) / A_i);
View
76 src/main/java/de/bwaldvogel/liblinear/SolverType.java
@@ -1,5 +1,9 @@
package de.bwaldvogel.liblinear;
+import java.util.HashMap;
+import java.util.Map;
+
+
public enum SolverType {
/**
@@ -7,59 +11,106 @@
*
* (fka L2_LR)
*/
- L2R_LR(true),
+ L2R_LR(0, true, false),
/**
* L2-regularized L2-loss support vector classification (dual)
*
* (fka L2LOSS_SVM_DUAL)
*/
- L2R_L2LOSS_SVC_DUAL(false),
+ L2R_L2LOSS_SVC_DUAL(1, false, false),
/**
* L2-regularized L2-loss support vector classification (primal)
*
* (fka L2LOSS_SVM)
*/
- L2R_L2LOSS_SVC(false),
+ L2R_L2LOSS_SVC(2, false, false),
/**
* L2-regularized L1-loss support vector classification (dual)
*
* (fka L1LOSS_SVM_DUAL)
*/
- L2R_L1LOSS_SVC_DUAL(false),
+ L2R_L1LOSS_SVC_DUAL(3, false, false),
/**
* multi-class support vector classification by Crammer and Singer
*/
- MCSVM_CS(false),
+ MCSVM_CS(4, false, false),
/**
* L1-regularized L2-loss support vector classification
*
* @since 1.5
*/
- L1R_L2LOSS_SVC(false),
+ L1R_L2LOSS_SVC(5, false, false),
/**
* L1-regularized logistic regression
*
* @since 1.5
*/
- L1R_LR(true),
+ L1R_LR(6, true, false),
/**
* L2-regularized logistic regression (dual)
*
* @since 1.7
*/
- L2R_LR_DUAL(true);
+ L2R_LR_DUAL(7, true, false),
+
+ /**
+ * L2-regularized L2-loss support vector regression (primal)
+ *
+ * @since 1.91
+ */
+ L2R_L1LOSS_SVR_DUAL(11, false, true),
+
+ /**
+ * L2-regularized L2-loss support vector regression (dual)
+ *
+ * @since 1.91
+ */
+ L2R_L2LOSS_SVR(12, false, true),
+
+ /**
+ * L2-regularized L1-loss support vector regression (dual)
+ *
+ * @since 1.91
+ */
+ L2R_L2LOSS_SVR_DUAL(13, false, true),
+
+ ;
private final boolean logisticRegressionSolver;
+ private final boolean supportVectorRegression;
+ private final int id;
- private SolverType( boolean logisticRegressionSolver ) {
+ private SolverType( int id, boolean logisticRegressionSolver, boolean supportVectorRegression ) {
+ this.id = id;
this.logisticRegressionSolver = logisticRegressionSolver;
+ this.supportVectorRegression = supportVectorRegression;
+ }
+
+ private static Map<Integer, SolverType> SOLVERS_BY_ID = new HashMap<Integer, SolverType>();
+ static {
+ for (SolverType solverType : SolverType.values()) {
+ SolverType old = SOLVERS_BY_ID.put(Integer.valueOf(solverType.getId()), solverType);
+ if (old != null) throw new Error("duplicate solver type ID: " + solverType.getId());
+ }
+ }
+
+ public int getId() {
+ return id;
+ }
+
+ public static SolverType getById(int id) {
+ SolverType solverType = SOLVERS_BY_ID.get(Integer.valueOf(id));
+ if (solverType == null) {
+ throw new RuntimeException("found no solvertype for id " + id);
+ }
+ return solverType;
}
/**
@@ -68,4 +119,11 @@ private SolverType( boolean logisticRegressionSolver ) {
public boolean isLogisticRegressionSolver() {
return logisticRegressionSolver;
}
+
+ /**
+ * @since 1.91
+ */
+ public boolean isSupportVectorRegression() {
+ return supportVectorRegression;
+ }
}
View
113 src/main/java/de/bwaldvogel/liblinear/Train.java
@@ -28,7 +28,10 @@ public static void main(String[] args) throws IOException, InvalidInputDataExcep
private Problem prob = null;
private void do_cross_validation() {
- int[] target = new int[prob.l];
+
+ double total_error = 0;
+ double sumv = 0, sumy = 0, sumvv = 0, sumyy = 0, sumvy = 0;
+ double[] target = new double[prob.l];
long start, stop;
start = System.currentTimeMillis();
@@ -36,37 +39,59 @@ private void do_cross_validation() {
stop = System.currentTimeMillis();
System.out.println("time: " + (stop - start) + " ms");
- int total_correct = 0;
- for (int i = 0; i < prob.l; i++)
- if (target[i] == prob.y[i]) ++total_correct;
-
- System.out.printf("correct: %d%n", total_correct);
- System.out.printf("Cross Validation Accuracy = %g%%%n", 100.0 * total_correct / prob.l);
+ if (param.solverType.isSupportVectorRegression()) {
+ for (int i = 0; i < prob.l; i++) {
+ double y = prob.y[i];
+ double v = target[i];
+ total_error += (v - y) * (v - y);
+ sumv += v;
+ sumy += y;
+ sumvv += v * v;
+ sumyy += y * y;
+ sumvy += v * y;
+ }
+ System.out.printf("Cross Validation Mean squared error = %g%n", total_error / prob.l);
+ System.out.printf("Cross Validation Squared correlation coefficient = %g%n", //
+ ((prob.l * sumvy - sumv * sumy) * (prob.l * sumvy - sumv * sumy)) / ((prob.l * sumvv - sumv * sumv) * (prob.l * sumyy - sumy * sumy)));
+ } else {
+ int total_correct = 0;
+ for (int i = 0; i < prob.l; i++)
+ if (target[i] == prob.y[i]) ++total_correct;
+
+ System.out.printf("correct: %d%n", total_correct);
+ System.out.printf("Cross Validation Accuracy = %g%%%n", 100.0 * total_correct / prob.l);
+ }
}
private void exit_with_help() {
System.out.printf("Usage: train [options] training_set_file [model_file]%n" //
+ "options:%n"
+ "-s type : set type of solver (default 1)%n"
- + " 0 -- L2-regularized logistic regression (primal)%n"
- + " 1 -- L2-regularized L2-loss support vector classification (dual)%n"
- + " 2 -- L2-regularized L2-loss support vector classification (primal)%n"
- + " 3 -- L2-regularized L1-loss support vector classification (dual)%n"
- + " 4 -- multi-class support vector classification by Crammer and Singer%n"
- + " 5 -- L1-regularized L2-loss support vector classification%n"
- + " 6 -- L1-regularized logistic regression%n"
- + " 7 -- L2-regularized logistic regression (dual)%n"
+ + " 0 -- L2-regularized logistic regression (primal)%n"
+ + " 1 -- L2-regularized L2-loss support vector classification (dual)%n"
+ + " 2 -- L2-regularized L2-loss support vector classification (primal)%n"
+ + " 3 -- L2-regularized L1-loss support vector classification (dual)%n"
+ + " 4 -- multi-class support vector classification by Crammer and Singer%n"
+ + " 5 -- L1-regularized L2-loss support vector classification%n"
+ + " 6 -- L1-regularized logistic regression%n"
+ + " 7 -- L2-regularized logistic regression (dual)%n"
+ + " 11 -- L2-regularized L2-loss epsilon support vector regression (primal)%n"
+ + " 12 -- L2-regularized L2-loss epsilon support vector regression (dual)%n"
+ + " 13 -- L2-regularized L1-loss epsilon support vector regression (dual)%n"
+ "-c cost : set the parameter C (default 1)%n"
+ + "-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)%n"
+ "-e epsilon : set tolerance of termination criterion%n"
- + " -s 0 and 2%n"
- + " |f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2,%n"
+ + " -s 0 and 2%n" + " |f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2,%n"
+ " where f is the primal function and pos/neg are # of%n"
- + " positive/negative data (default 0.01)%n"
- + " -s 1, 3, 4 and 7%n"
- + " Dual maximal violation <= eps; similar to libsvm (default 0.1)%n"
+ + " positive/negative data (default 0.01)%n" + " -s 11%n"
+ + " |f'(w)|_2 <= eps*|f'(w0)|_2 (default 0.001)%n"
+ + " -s 1, 3, 4 and 7%n" + " Dual maximal violation <= eps; similar to libsvm (default 0.1)%n"
+ " -s 5 and 6%n"
+ " |f'(w)|_1 <= eps*min(pos,neg)/l*|f'(w0)|_1,%n"
+ " where f is the primal function (default 0.01)%n"
+ + " -s 12 and 13\n"
+ + " |f'(alpha)|_1 <= eps |f'(alpha0)|,\n"
+ + " where f is the dual function (default 0.1)\n"
+ "-B bias : if bias >= 0, instance x becomes [x; bias]; if < 0, no bias term added (default -1)%n"
+ "-wi weight: weights adjust the parameter C of different classes (see README for details)%n"
+ "-v n: n-fold cross validation mode%n"
@@ -91,7 +116,7 @@ void parse_command_line(String argv[]) {
int i;
// eps: see setting below
- param = new Parameter(SolverType.L2R_L2LOSS_SVC_DUAL, 1, Double.POSITIVE_INFINITY);
+ param = new Parameter(SolverType.L2R_L2LOSS_SVC_DUAL, 1, Double.POSITIVE_INFINITY, 0.1);
// default values
bias = -1;
cross_validation = false;
@@ -102,11 +127,14 @@ void parse_command_line(String argv[]) {
if (++i >= argv.length) exit_with_help();
switch (argv[i - 1].charAt(1)) {
case 's':
- param.solverType = SolverType.values()[atoi(argv[i])];
+ param.solverType = SolverType.getById(atoi(argv[i]));
break;
case 'c':
param.setC(atof(argv[i]));
break;
+ case 'p':
+ param.setP(atof(argv[i]));
+ break;
case 'e':
param.setEps(atof(argv[i]));
break;
@@ -151,13 +179,30 @@ void parse_command_line(String argv[]) {
}
if (param.eps == Double.POSITIVE_INFINITY) {
- if (param.solverType == SolverType.L2R_LR || param.solverType == SolverType.L2R_L2LOSS_SVC) {
- param.setEps(0.01);
- } else if (param.solverType == SolverType.L2R_L2LOSS_SVC_DUAL || param.solverType == SolverType.L2R_L1LOSS_SVC_DUAL
- || param.solverType == SolverType.MCSVM_CS || param.solverType == SolverType.L2R_LR_DUAL) {
- param.setEps(0.1);
- } else if (param.solverType == SolverType.L1R_L2LOSS_SVC || param.solverType == SolverType.L1R_LR) {
- param.setEps(0.01);
+ switch (param.solverType) {
+ case L2R_LR:
+ case L2R_L2LOSS_SVC:
+ param.setEps(0.01);
+ break;
+ case L2R_L2LOSS_SVR:
+ param.setEps(0.001);
+ break;
+ case L2R_L2LOSS_SVC_DUAL:
+ case L2R_L1LOSS_SVC_DUAL:
+ case MCSVM_CS:
+ case L2R_LR_DUAL:
+ param.setEps(0.1);
+ break;
+ case L1R_L2LOSS_SVC:
+ case L1R_LR:
+ param.setEps(0.01);
+ break;
+ case L2R_L1LOSS_SVR_DUAL:
+ case L2R_L2LOSS_SVR_DUAL:
+ param.setEps(0.1);
+ break;
+ default:
+ throw new IllegalStateException("unknown solver type: " + param.solverType);
}
}
}
@@ -170,7 +215,7 @@ void parse_command_line(String argv[]) {
*/
public static Problem readProblem(File file, double bias) throws IOException, InvalidInputDataException {
BufferedReader fp = new BufferedReader(new FileReader(file));
- List<Integer> vy = new ArrayList<Integer>();
+ List<Double> vy = new ArrayList<Double>();
List<Feature[]> vx = new ArrayList<Feature[]>();
int max_index = 0;
@@ -191,7 +236,7 @@ public static Problem readProblem(File file, double bias) throws IOException, In
}
try {
- vy.add(atoi(token));
+ vy.add(atof(token));
} catch (NumberFormatException e) {
throw new InvalidInputDataException("invalid label: " + token, file, lineNr, e);
}
@@ -265,7 +310,7 @@ void readProblem(String filename) throws IOException, InvalidInputDataException
return newArray;
}
- private static Problem constructProblem(List<Integer> vy, List<Feature[]> vx, int max_index, double bias) {
+ private static Problem constructProblem(List<Double> vy, List<Feature[]> vx, int max_index, double bias) {
Problem prob = new Problem();
prob.bias = bias;
prob.l = vy.size();
@@ -283,9 +328,9 @@ private static Problem constructProblem(List<Integer> vy, List<Feature[]> vx, in
}
}
- prob.y = new int[prob.l];
+ prob.y = new double[prob.l];
for (int i = 0; i < prob.l; i++)
- prob.y[i] = vy.get(i);
+ prob.y[i] = vy.get(i).doubleValue();
return prob;
}
View
10 src/main/java/de/bwaldvogel/liblinear/Tron.java
@@ -2,7 +2,9 @@
import static de.bwaldvogel.liblinear.Linear.info;
-
+/**
+ * Trust Region Newton Method optimization
+ */
class Tron {
private final Function fun_obj;
@@ -101,15 +103,15 @@ else if (actred < eta2 * prered)
if (gnorm <= eps * gnorm1) break;
}
if (f < -1.0e+32) {
- info("warning: f < -1.0e+32%n");
+ info("WARNING: f < -1.0e+32%n");
break;
}
if (Math.abs(actred) <= 0 && prered <= 0) {
- info("warning: actred and prered <= 0%n");
+ info("WARNING: actred and prered <= 0%n");
break;
}
if (Math.abs(actred) <= 1.0e-12 * Math.abs(f) && Math.abs(prered) <= 1.0e-12 * Math.abs(f)) {
- info("warning: actred and prered too small%n");
+ info("WARNING: actred and prered too small%n");
break;
}
}
View
35 src/test/java/de/bwaldvogel/liblinear/LinearTest.java
@@ -57,7 +57,7 @@ public static Problem createRandomProblem(int numClasses) {
prob.l = random.nextInt(100) + 1;
prob.n = random.nextInt(100) + 1;
prob.x = new FeatureNode[prob.l][];
- prob.y = new int[prob.l];
+ prob.y = new double[prob.l];
for (int i = 0; i < prob.l; i++) {
@@ -104,7 +104,7 @@ public void testTrainPredict() {
prob.x[3][1] = new FeatureNode(2, 1);
prob.x[3][2] = new FeatureNode(4, 1);
- prob.y = new int[4];
+ prob.y = new double[4];
prob.y[0] = 0;
prob.y[1] = 1;
prob.y[2] = 1;
@@ -117,7 +117,11 @@ public void testTrainPredict() {
if (C < 0.2) if (solver == SolverType.L1R_L2LOSS_SVC) continue;
if (C < 0.7) if (solver == SolverType.L1R_LR) continue;
- Parameter param = new Parameter(solver, C, 0.1);
+ if (solver.isSupportVectorRegression()) {
+ continue;
+ }
+
+ Parameter param = new Parameter(solver, C, 0.1, 0.1);
Model model = Linear.train(prob, param);
double[] featureWeights = model.getFeatureWeights();
@@ -128,14 +132,14 @@ public void testTrainPredict() {
}
int i = 0;
- for (int value : prob.y) {
- int prediction = Linear.predict(model, prob.x[i]);
- assertThat(prediction).isEqualTo(value);
+ for (double value : prob.y) {
+ double prediction = Linear.predict(model, prob.x[i]);
+ assertThat(prediction).as("prediction with solver " + solver).isEqualTo(value);
if (model.isProbabilityModel()) {
double[] estimates = new double[model.getNrClass()];
- int probabilityPrediction = Linear.predictProbability(model, prob.x[i], estimates);
+ double probabilityPrediction = Linear.predictProbability(model, prob.x[i], estimates);
assertThat(probabilityPrediction).isEqualTo(prediction);
- assertThat(estimates[probabilityPrediction]).isGreaterThanOrEqualTo(1.0 / model.getNrClass());
+ assertThat(estimates[(int)probabilityPrediction]).isGreaterThanOrEqualTo(1.0 / model.getNrClass());
double estimationSum = 0;
for (double estimate : estimates) {
estimationSum += estimate;
@@ -157,10 +161,10 @@ public void testCrossValidation() throws Exception {
Parameter param = new Parameter(SolverType.L2R_LR, 10, 0.01);
int nr_fold = 10;
- int[] target = new int[prob.l];
+ double[] target = new double[prob.l];
Linear.crossValidation(prob, param, nr_fold, target);
- for (int clazz : target) {
+ for (double clazz : target) {
assertThat(clazz).isGreaterThanOrEqualTo(0).isLessThan(numClasses);
}
}
@@ -194,7 +198,7 @@ public void testTrainUnsortedProblem() {
prob.x[0][0] = new FeatureNode(2, 1);
prob.x[0][1] = new FeatureNode(1, 1);
- prob.y = new int[4];
+ prob.y = new double[4];
prob.y[0] = 0;
Parameter param = new Parameter(SolverType.L2R_LR, 10, 0.1);
@@ -212,13 +216,14 @@ public void testTrainTooLargeProblem() {
prob.l = 1000;
prob.n = 20000000;
prob.x = new FeatureNode[prob.l][];
- prob.y = new int[prob.l];
+ prob.y = new double[prob.l];
for (int i = 0; i < prob.l; i++) {
prob.x[i] = new FeatureNode[] {};
prob.y[i] = i;
}
for (SolverType solverType : SolverType.values()) {
+ if (solverType.isSupportVectorRegression()) continue;
Parameter param = new Parameter(solverType, 10, 0.1);
try {
Linear.train(prob, param);
@@ -341,7 +346,7 @@ public void testTranspose() throws Exception {
prob.x[3][1] = new FeatureNode(3, 1);
prob.x[3][2] = new FeatureNode(4, 1);
- prob.y = new int[4];
+ prob.y = new double[4];
prob.y[0] = 0;
prob.y[1] = 1;
prob.y[2] = 1;
@@ -438,7 +443,7 @@ public void testTranspose2() throws Exception {
prob.x[4][0] = new FeatureNode(3, 1);
prob.x[4][1] = new FeatureNode(10, 3);
- prob.y = new int[5];
+ prob.y = new double[5];
prob.y[0] = 0;
prob.y[1] = 1;
prob.y[2] = 1;
@@ -518,7 +523,7 @@ public void testTranspose3() throws Exception {
Problem prob = new Problem();
prob.l = 3;
prob.n = 4;
- prob.y = new int[3];
+ prob.y = new double[3];
prob.x = new FeatureNode[4][];
prob.x[0] = new FeatureNode[3];
prob.x[1] = new FeatureNode[4];
View
15 src/test/java/de/bwaldvogel/liblinear/TrainTest.java
@@ -19,18 +19,23 @@ public void testParseCommandLine() {
Train train = new Train();
for (SolverType solver : SolverType.values()) {
- train.parse_command_line(new String[] {"-B", "5.3", "-s", "" + solver.ordinal(), "model-filename"});
+ train.parse_command_line(new String[] {"-B", "5.3", "-s", "" + solver.getId(), "-p", "0.01", "model-filename"});
Parameter param = train.getParameter();
assertThat(param.solverType).isEqualTo(solver);
// check default eps
- if (solver.ordinal() == 0 || solver.ordinal() == 2 //
- || solver.ordinal() == 5 || solver.ordinal() == 6) {
+ if (solver.getId() == 0 || solver.getId() == 2 //
+ || solver.getId() == 5 || solver.getId() == 6) {
assertThat(param.eps).isEqualTo(0.01);
+ } else if (solver.getId() == 7) {
+ assertThat(param.eps).isEqualTo(0.1);
+ } else if (solver.getId() == 11) {
+ assertThat(param.eps).isEqualTo(0.001);
} else {
assertThat(param.eps).isEqualTo(0.1);
}
// check if bias is set
assertThat(train.getBias()).isEqualTo(5.3);
+ assertThat(param.p).isEqualTo(0.01);
}
}