HiveCote.main() results in java.lang.UnsupportedOperationException

[davcem]

When I start the main function for the HIVE-Cote classifier (Master c3e7215) it results in the following exception:
training (group a): TSF
java.lang.UnsupportedOperationException: getTrainAcc not implemented in class timeseriesweka.classifiers.interval_based.TSF
at timeseriesweka.classifiers.TrainAccuracyEstimator.getTrainAcc(TrainAccuracyEstimator.java:72)
at timeseriesweka.classifiers.hybrids.HiveCote.buildClassifier(HiveCote.java:193)
at at.outfisltisl.classification.AbstractClassifierWrapper.performStep(AbstractClassifierWrapper.java:69)
at at.outfisltisl.pipeline.Pipeline.runPipeline(Pipeline.java:114)
at at.outfisltisl.app.RunClassificationComparison.runSingleClassifierExperiment(RunClassificationComparison.java:57)
at at.outfisltisl.app.RunClassificationComparison.runExperimentsForDataset(RunClassificationComparison.java:40)
at at.outfisltisl.app.OutfisltislApp.main(OutfisltislApp.java:341)

The problem here is that the function getTrainAcc() is not implemented for timeseriesweka.classifiers.interval_based.TSF.java
btw: I like the usage of default method within interface timeseriesweka.classifiers.TrainAccuracyEstimator ;-)

[TonyBagnall]

that is odd, and bad. I'll look into it, as that is an interface method

[davcem]

If you look into timeseriesweka.classifiers.interval_based.TSF.java it seems that trainAccuracy (getTrainAcc()) is already described starting in Line 155.

[TonyBagnall]

ok, so the TSF problem is easy, but have recreated the ShapeletTransform one for both Flat and Hive. Will try fix it tonight

[TonyBagnall]

so both these cote versions are legacy ones, we have restructured the ensembles and have not yet refactored the latest version of HIVE-COTE, which is here. I'll fix and check the other two, but can you try this one?
https://github.com/uea-machine-learning/tsml/blob/master/src/main/java/weka_extras/classifiers/ensembles/HIVE_COTE.java

[davcem]

Seems to be a tougher one, ...

HIVE_COTE from master(c3e7215):
Exception in thread "main" java.lang.IllegalArgumentException: Src and Dest differ in # of attributes: 301 != 1
at weka.core.RelationalLocator.copyRelationalValues(RelationalLocator.java:87)
at weka.filters.Filter.copyValues(Filter.java:371)
at weka.filters.Filter.push(Filter.java:288)
at weka.filters.SimpleBatchFilter.batchFinished(SimpleBatchFilter.java:257)
at weka.filters.Filter.useFilter(Filter.java:682)
at weka_extras.classifiers.ensembles.AbstractEnsemble.buildClassifier(AbstractEnsemble.java:890)
at weka_extras.classifiers.ensembles.AbstractEnsemble.lambda$0(AbstractEnsemble.java:375)
at weka_extras.classifiers.ensembles.AbstractEnsemble.trainModules(AbstractEnsemble.java:401)
at weka_extras.classifiers.ensembles.AbstractEnsemble.initialiseModules(AbstractEnsemble.java:349)
at weka_extras.classifiers.ensembles.AbstractEnsemble.buildClassifier(AbstractEnsemble.java:899)
at weka_extras.classifiers.ensembles.HIVE_COTE.buildClassifier(HIVE_COTE.java:135)
at weka_extras.classifiers.ensembles.HIVE_COTE.main(HIVE_COTE.java:205)

with this main:

public static void main(String[] args) throws Exception{
	String datasetName = "MoteStrain";
	HIVE_COTE hive = new HIVE_COTE();
	Instances train = DatasetLoading.loadDataNullable("/home/raul/git/outfisltisl/data/"+datasetName+"/"+datasetName + "_TRAIN");
	Instances test = DatasetLoading.loadDataNullable("/home/raul/git/outfisltisl/data/"+datasetName+"/"+datasetName+"_TEST");
	         
	hive.buildClassifier(train);
}

but this is due to the ShapeletTransform issue (#151)

HIVE_COTE from master with "ShapeletTransform" hack and same main(), results in:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at evaluation.storage.ClassifierResults.findMedianPredTime(ClassifierResults.java:1976)
at evaluation.storage.ClassifierResults.findAllStats(ClassifierResults.java:1769)
at evaluation.storage.ClassifierResults.findAllStatsOnce(ClassifierResults.java:1789)
at weka_extras.classifiers.ensembles.AbstractEnsemble.initialiseModules(AbstractEnsemble.java:358)
at weka_extras.classifiers.ensembles.AbstractEnsemble.buildClassifier(AbstractEnsemble.java:904)
at weka_extras.classifiers.ensembles.HIVE_COTE.buildClassifier(HIVE_COTE.java:135)
at weka_extras.classifiers.ensembles.HIVE_COTE.main(HIVE_COTE.java:205)

HIVE_COTE provided by you (https://github.com/uea-machine-learning/tsml/blob/master/src/main/java/weka_extras/classifiers/ensembles/HIVE_COTE.java) with "ShapeletTransform" hack and same main(), results in:

module: EE
**getAcc():calculateAcc() no true class values supplied yet, cannot calculate accuracy
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at evaluation.storage.ClassifierResults.findMedianPredTime(ClassifierResults.java:1976)
at evaluation.storage.ClassifierResults.findAllStats(ClassifierResults.java:1769)
at evaluation.storage.ClassifierResults.findAllStatsOnce(ClassifierResults.java:1789)
at weka_extras.classifiers.ensembles.AbstractEnsemble.initialiseModules(AbstractEnsemble.java:358)
at weka_extras.classifiers.ensembles.AbstractEnsemble.buildClassifier(AbstractEnsemble.java:904)
at weka_extras.classifiers.ensembles.HIVE_COTE.buildClassifier(HIVE_COTE.java:135)
at weka_extras.classifiers.ensembles.HIVE_COTE.main(HIVE_COTE.java:205)

I added System.out.println("module: " + module.moduleName); to the EnsembleMoudle loop in AbstractEnsemble.initialiseModules() which blames module "EE" = ElasticEnsemble as the last module before the exception...Hope this helps

[TonyBagnall]

ok, I see whats going on, but this will require some re-engineering and we are all at a conference this week. Will try sort it this week but cannot promise. The reason this is broken is we never build cote in one go. This code has suffered because our ensemble structure has changed and we had not realised it effected this code. We always build the components then assemble afterwards from the output files. The easiest way to do this is by using the Experiments.java class (which is what we always use). I will post updates. Thanks again for highlighting this

[davcem]

For me no hurry is required, I really appreciate your efforts. I was already able to integrate tsml library into my project (including ShapeletTransform hack) and as I mentioned FlatCote is working. Nevertheless I am looking forward to an update, and also are willing to test/review it :-)

[TonyBagnall]

HiveCote should work as advertised now, I'll close this, please raise a new issue if you encounter further problems.