[ENH] New similarity search module #724
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rimental/similarity_search
…an distance (#756) * Adding TopKSimilarity Search class with euclidean distance profiles * Removing old normalization attributes * Removing old normalization attributes in predict --------- Co-authored-by: MatthewMiddlehurst <m.middlehurst@uea.ac.uk>
…eon-toolkit/aeon into experimental/similarity_search
TonyBagnall
changed the title
TonyBagnall
commented
Oct 6, 2023
…eon-toolkit/aeon into experimental/similarity_search
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…heck, some utils functions
hadifawaz1999
previously approved these changes
Oct 25, 2023
MatthewMiddlehurst
added
enhancement
There was a problem hiding this comment.
I have no issue with introducing the module, I have not really looked at the specifics of how it is implemented and am content to let it evolve as usage and implementations increase.
Most of the comments are regarding the documentation, I did not check the examples but may come back to do so (or make a PR later).
aeon/similarity_search/distance_profiles/tests/test_naive_euclidean.py
Outdated
Show resolved
Hide resolved
baraline
approved these changes
Oct 28, 2023
MatthewMiddlehurst
approved these changes
Oct 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this PR introduces a new (experimental) module: similarity search by @baraline
Overview of the content:
Changed base shape expected for X to (n_samples, n_channels, n_timestamps), and for Q to (n_channels, q_length) to handle the more general case.
The choice of how a candidate of X is matched with Q is left to the child classes (e.g. TopKSimilaritySearch class will ask for a k argument and return top-k matches). A possibility would be to make a base class for distance profiles to force them having the same API and have attributes indicating if they are normalized or not.
Normalization happens in the predict method every time it is called. It stores the mean and std of each channels of Q and of all possible candidates of X as attributes (e.g _X_means). They are stored as argument to be called in the child classes of the base class when the distance profile is normalized. It may be possible to avoid recomputing it by checking if the length of Q and the stored input _X have not changed.
Added naive Euclidean and normalized naive Euclidean distance profiles with basic tests
Added top-k similarity search class with basic tests. The return of predict is an array of tuple as (with k=1) [(id_sample, id_timestamp)]
TODOs left for future PR :
Make documentation for module and estimator
Decide how to handle/check when there is not input or query length change to avoid computing normalization again
Refine testing scenarios and add them to the CI pipeline (if not made automaticaly ?)
Add non-naïve Euclidean distance profile with Mueen Algorithm.
Add a template for new cases