Search: Keywords filter should not exclude stopwords

[joachimtingvold]

What does not work as expected?

Adding meme or memes as description and keyword for a picture. Trying to search with meme or memes as query string has zero results. Trying to search with keywords:meme or keywords:memes returns a whole lot of images (it seems to be all images, but have not verified). Re-index has no effect.

How can we reproduce it?

1. Add "meme" and/or "memes" as description and keyword(s).
2. Click on "Search"
3. Search for meme or memes, and it shows zero results
4. Search for keywords:meme or keywords:memes, and it shows many results

What behavior do you expect?

It should show images having meme or memes as description and/or keyword when searching for meme or memes (and not zero, as it does now).

It should show only images having meme or memes as keyword when searching for keywords:meme or keywords:memes (and not "all" as it does now).

More observations

Trying to use kkthxbye as the description and keyword, it seems to work as expected when searching for kkthxbye or keywords:kkthxbye (i.e. only the one image shows up in both searches).

It also shows results for a substring-search, down to 4 characters. If you try to use kkt, it shows no images, and if you try to use keywords:kkt, it shows "all" images again.

There seems to be an exception to the "shorter than 4 characters" in a search, and that is if it has a label with fewer characters (for example "cat"), which then also is added as a keyword. Search for cat and keywords:cat both return only the actual pictures with those labels/keywords. Trying kkk as a manual keyword did not work, but fun works.

Software

Photoprism 211215-93b26f19-Linux-x86_64
Chrome version 96.0.4664.110 (Official Build) (x86_64)
MacOS 11.6.2

[joachimtingvold]

[lastzero]

Might be on the full text stoplist in the txt package. Not sure why the word has been added, we didn't create the lists on our own. In that case, the code works as expected and you just need to remove these.

[joachimtingvold]

You seem to be correct. Both meme and memes are present in pkg/txt/stopwords.go.

However, that also seems to be directly related to why keywords:<string>-searches yields "all" images as result (if <string> is a word from the stoplist), even though none of the images matches that search. Is that also expected behaviour?

[lastzero]

We could add code that returns an empty result if all words are stopwords. When there are other filters, they must be ignored.

[joachimtingvold]

Yeah, that would make more sense. If you have large amounts of photos, it's hard to differentiate between "many photos that actually matches the search query" and "all images returned by default due to random stopwords".

[lastzero]

Meme and memes have been removed from the stopwords list. When keywords contain stopwords only, an empty result is returned with an error ("invalid request"). We can add a more helpful message later if needed (needs translation).

A new Development Preview is available for testing soon.