v4.0.0

[zpzim]

This release will be the first one officially supporting pyscamp distributions on conda-forge. Shortly after this release goes live, you should be able to install SCAMP via:

conda install pyscamp-gpu -c conda-forge to install a CPU/GPU enabled pyscamp on Windows and Linux.

conda install pyscamp-cpu -c conda-forge to install a CPU-only version of pyscamp on Windows, Linux, and MacOS.

This release introduces a couple of breaking changes needed in order to support these distributions and further development on SCAMP.

1. CUDA 11.0 or higher is required to build SCAMP now.
2. The minimum cmake requirement has been increased to 3.18

What's Changed

• Update SCAMP to require CUDA 11.0 or greater. by @zpzim in Update SCAMP to require CUDA 11.0 or greater. #109
• Update docs with information on conda package availability by @zpzim in Update docs with information on conda package availability #111

Full Changelog: v3.0.3...v4.0.0

---

This discussion was created from the release v4.0.0.

[hchaudhary1]

Does anyone know of good tools to post-process matrix profile generated?

[zpzim]

What are you trying to do with the matrix profile? What kind of post processing do you need to do?

[zpzim]

I see, so you want to identify the repeating motifs themselves? The matrix profile allows you to do this, you can directly extract the motifs from the input data corresponding with the smallest matrix profile values. These are the patterns which are most correlated to another pattern in the data.

Note that this does not handle frequency; a pattern only has to match once to have a low matrix profile value. If you are looking to find the patterns in the data which are correlated most frequently. You can do one of two things:

1. Use the knn matrix profile to identify the top k matches for each subsequence. However, you will need to do some post processing here to figure out which patterns are the most well connected to others. You essentially are producing a graph, where nodes are subsequences and correlations between subsequences are edges. And maybe you want to look for cliques in this graph to find highly connected patterns? I have written some code in the past to do something similar but I'd need to go dig it up, not sure if I still have it. But you should be able to use existing graph libraries to do something similar. You can even visualize the graphs with something like graphviz.

2. You could try to use the "sum" matrix profile (see SCAMP documentation on profile types) with a high threshold to generate a sort of frequency histogram for each pattern. The largest values in the output would be the most frequent patterns.

Do either of those ideas make sense in the context of what you are doing? Is this the kind of post processing you want?

[hchaudhary1]

k-NN was what i was originally thinking about. But the SUM method seems interesting, I'll check it out. thanks

Also, I was thinking to do an FFT on motifs, and then use UMAP to cluster them together. But I don't want to build such tools, if there is already available projects to handle this.

Finally, I also wanted to look at DTW, or SWAMP based algo. Or I guess, I can manually scale the time series from 50% to 200%, and cross check.