Search: Omit full text index if query is too short

[alexislefebvre]

Examples:

• 01234 : https://demo.photoprism.org/browse?view=cards&order=added&q=01234&public=true&quality=3
• zz01: https://demo.photoprism.org/browse?view=cards&order=added&q=zz01&public=true&quality=3
• zzz01: https://demo.photoprism.org/browse?view=cards&order=added&q=zzz01&public=true&quality=3

But zzzz01 returns 0 result: https://demo.photoprism.org/browse?view=cards&order=added&q=zzzz01&public=true&quality=3

[lastzero]

Stopwords, one or two letter Latin "words" and purely numeric values will be ignored when searching because they are not indexed (for the obvious reason). So it's not known if there are results for those or not. Any specific reason you search for zz01?

[alexislefebvre]

Stopwords, one or two letter Latin "words" and purely numeric values will be ignored when searching because they are not indexed (for the obvious reason). So it's not known if there are results for those or not.

Could you please explain the obvious reason? Why ignore the numbers?

Any specific reason you search for zz01?

I restarted the indexing to detect faces, I noticed a filename with only numbers in the logs, so I wanted to see that file, so I searched it. The tests with zz… were to try to understand what was happening. I could have used the filters name or original though.

[lastzero]

Because it's a FULL TEXT index, we index words, not numbers. You can find single letters or numbers in almost any file, so it's not useful for filtering / searching and blows up your index / database on top of it, leading to bad performance. If you search for specific fields like the file name, this does not apply.

[lastzero]

We could in fact automatically do a filename wildcard search for queries like zz01. Would that make sense and match your expectations?

[alexislefebvre]

We can already use the name filter for searching by filename, I think it's the good way. And I think that doing automatic search would be confusing for users.

I rephrased my request: As an end user, if I search something and it doesn't match with photos (because of the way the database is indexed), I expect to see a message like “No results” and/or a message that explain why it's invalid.

[lastzero]

We don't know if it matches because there is no index. Compare to MySQL:

The default minimum length of words that are found by full-text searches is three characters for InnoDB search indexes, or four characters for MyISAM

[lastzero]

In fact we had a "query too short" error, but the limitation specifically applies to Latin characters: There is no length limitation for Chinese etc

[alexislefebvre]

• zz01 shows no result ✔️ : https://demo.photoprism.org/browse?view=cards&order=added&q=zz01&public=true&quality=3
• zzz01 shows all results ❌ : https://demo.photoprism.org/browse?view=cards&order=added&q=zzz01&public=true&quality=3
• zzzz01 shows no result ✔️ : https://demo.photoprism.org/browse?view=cards&order=added&q=zzzz01&public=true&quality=3

I think that there's still an issue for end users: is a string with 5 characters treated differently?

[lastzero]

True, because zzz is long enough to be a word, but it's on our stoplist as it's not a word - so the index couldn't contain it, even when it exists. We could also detect this special case and look in file names only in a future release.

[alexislefebvre]

Thanks for your answer, ali show all the results but bob doesn't. Because it's in that list?

[lastzero]

Started a new Development Preview build including the following changes:

• People's names will be added to the full text index in the background and when editing photos.
• Keep in mind index updates may take a while, they don't happen in real time.
• Using the full text index means names matching a stopword like "img" won't be found.
• If the query is too short for the full text index or matches a stopword, a photo name wildcard search will be performed instead.