[ENH] feature set benchmarks after Fulcher et al for TSC

[fkiraly]

It would be really great if we could get sktime to a state where we can easily replicate standard feature set benchmarks (e.g., time series classification) as presented by @benfulcher.

Example experiment:
using the algorithm [transformer][1-knn classif] or [transformer][linear svm], compute/compare performance as supervised time series classifier on the UCI data set repository.

Relevant feature sets, useable as transformers, are listed in #2982

Caveat mentioned by @benfulcher: catch-22 should not be used in isolation on the UCI repository (or not only), since it does not contain summary features. Better option: catch-24 = catch-22 plus mean, var; or, similar: feature union of catch-22 transformer and SummaryTransformer.

FYI @ltsaprounis, @DBCerigo, @TNTran92, interesting use case for kotsu and/or benchmarking work?

[fkiraly]

FYI @alex-hh

[benfulcher]

Some feature sets (constructed to focus only on properties of dynamics, and which are therefore insensitive to rescalings, and thus shifts in mean or variance) would be put at a disadvantage on problems in which there are trivial class differences (like class 1 has a higher mean than class 2). pycatch22 has an option to set catch24=True and include two simple additional features (mean + std) for problems in which this may be class-informative.

[fkiraly]

pycatch22 has an option to set catch24=True and include two simple additional features (mean + std) for problems in which this may be class-informative.

Thanks for reminding us 😄. SummaryTransformer has mean, std, and quantile features, so that should also work if in FeatureUnion with catch22.