Problem: substring searches via Elasticsearch often fail to bring up files and SIPs

[sevein]

Misty originally raised this issue and suggested a solution here: artefactual/archivematica#364. I closed that one because it's not being actively developed by the original author. We may want to re-open it or submit a new PR instead.

Misty explained the issue better than I could do it:

The default tokenization used by Elasticsearch doesn't deal well with the ways that path strings are constructed; substring searches often fail to bring up files and SIPs which would be expected by users. For example, trying to search for "events" will fail to bring up files in a SIP named "Events_2005-d1c6c1a2-f613-497f-a527-64eb6861f03c".

CC @jraddaoui - you may want to look into this as part of the work that you're doing for artefactual/archivematica#1171.

[sallain]

Added this as a child of #273, which is the epic for all search-related issues.

[jraddaoui]

This has been addressed in the ES upgrade by adding a new analyzer/tokenizer for filepath fields. Those paths should now be split in the following chars: - _ . / \.

[sallain]

This works!!!!!! 🎉 🌮 🍾

If I have a filename like View_from_lookout_over_Queenstown_towards_the_Remarkables_in_spring.jpg, I can search for any of the words in the filename to find my file. Searching for a partial word like Queens doesn't display my result, but adding a wildcard Queens* and using the Phrase parameter (as opposed to Keyword) does work - this is the expected behaviour.

There was a change in the search behaviour related to the availability of the raw JSON data; that has been described in #274 (comment). I'll verify this issue and keep #274 open to deal with that issue.