Problem with learning and saving classifier without performing any testing

[niedakh]

I am trying to learn a classifier and store it for the future without the need to test it at the time of classification. I am using the following command to make sure no splitting for test is done (without it meka reported test instance count larger than 0):

/usr/bin/java -cp "/home/niedakh/scikit/meka/meka-release-1.9.0/lib/*" meka.classifiers.multilabel.LC -W weka.classifiers.bayes.NaiveBayes -threshold 0 -verbosity 5 -split-percentage 100 -t ~/engine/scikit-multilearn/meka/data/scene-train.arff -d classifier.dump

I am receiving an error:

java.lang.ArrayIndexOutOfBoundsException: 0
    at meka.core.MLEvalUtils.getMLStats(MLEvalUtils.java:57)
    at meka.core.Result.getStats(Result.java:289)
    at meka.classifiers.multilabel.Evaluation.evaluateModel(Evaluation.java:263)
    at meka.classifiers.multilabel.Evaluation.runExperiment(Evaluation.java:187)
    at meka.classifiers.multilabel.ProblemTransformationMethod.runClassifier(ProblemTransformationMethod.java:172)
    at meka.classifiers.multilabel.ProblemTransformationMethod.evaluation(ProblemTransformationMethod.java:152)
    at meka.classifiers.multilabel.LC.main(LC.java:148)

[joergwicker]

Meka tried to do an evaluation in any case. And it ended up with an empty test set, and empty results. As a quick solution, I added a check if the train or test set is empty, then Meka does not evaluate and just trains on the full set. It is pushed it to the 1.9.1.-SNAPSHOT. But we might change the options, add a flag for evaluation or only training.

[jmread]

That's a good point actually. Nice to have that fix, Joerg. Another quick
work-around for 1.9.0 is to simply specify the dataset again with the -T
flag (for test set). For example, -t dataset.arff -d classifier.dump -T
dataset.arff. Meka will train on the full dataset.arff and then dump the
classifier to disk. You can ignore the evaluation, and use a different test
set when you load the classifier from disk again. But better to have the
fix :-)

On 29 February 2016 at 01:03, Joerg Wicker notifications@github.com wrote:

Meka tried to do an evaluation in any case. And it ended up with an empty
test set, and empty results. As a quick solution, I added a check if the
train or test set is empty, then Meka does not evaluate and just trains on
the full set. It is pushed it to the 1.9.1.-SNAPSHOT. But we might change
the options, add a flag for evaluation or only training.

—
Reply to this email directly or view it on GitHub
#1 (comment).

[fracpete]

Closing this issue mentioning that you can use the -no-eval flag to suppress evaluation.