Metadata: Preserve stopwords in existing keywords

[graciousgrey]

Current behaviour:
If an image has flash and keyword information in exif. Only flash is set as keyword, not the keywords itself.

Related code:
internal/meta/exif.go

Related test:
internal/photoprism/index_mediafile_test.go

Expected behaviour:
Both flash and the keywords MUST be added as keywords

[uLUViL3gCs]

I can confirm same behavior.
I'm in process to import my whole photo DB into PP and as such I'm running dry-runs with a random selection to see that none information (including keywords) gets lost.
For some of my photos, none of its keywords got reflected, while for others all of the keywords were shown as desired.

I am glad to see this bug report, as I miserably failed in figuring out the trigger of this event.
Having checked the failed photos, I can confirm that all of those have "flash" keyword reflected in PP.

Here a shortened summary of exif flash info of the impacted photos:

xxx@yyy:~/Pics$ exiftool -all -a -G0:1 -s images/* | grep -i flash
[EXIF:ExifIFD] Flash : On, Fired
[EXIF:ExifIFD] FlashpixVersion : 0100
[EXIF:ExifIFD] LightSource : Flash
[EXIF:ExifIFD] Flash : Auto, Fired
[EXIF:ExifIFD] Flash : Auto, Fired, Red-eye reduction

Thanks for reporting the bug and all your efforts.

I'm running PhotoPrism® 210222-ac5a9d5e-Linux-x86_64 on Mac OS.

[lastzero]

This is part of our latest Development Preview and needs comprehensive testing before we can release it:

https://docs.photoprism.org/release-notes/

[uLUViL3gCs]

I started verifying this fix with those pictures that I stumbled across earlier and from what I can say so far, the fix seems to work for at least those two scenarios:
I) related files are getting imported to a PP version having the fix
II) related files that got imported on PP version having #1153 and being re-indexed on PP version proving the fix (complete Rescan)
I am going to continue importing photos next days/weeks and will report failures if and when occuring...

What I have done:
Scenario I)

1. take some jpg files which do have “flash” as keyword, as well as some other keywords
2. trigger import on PP ":latest" (210422-97e75b04-Linux-x86_64)
3. verify keywords
   ==> NOK:
   — keyword “flash” is shown (OK)
   — PP generated keywords shown (OK)
   — remaining additional keywords from source file not shown (NOK)
4. trigger import of files from step 1) on a “fresh” (ie. new parent location for storage/originals/DB) ":preview" (210425-dd652ce4-Linux-x86_64)
5. verify keywords
   ==> OK:
   — keyword “flash” is shown (OK)
   — PP generated keywords shown (OK)
   — remaining additional keywords from source file shown as well (OK)
   ====>> Bug fix OK

Scenario II)

10. start with a “fresh” PP (ie. new parent location for storage/originals/DB) ":latest" (210422-97e75b04-Linux-x86_64)
11. repeat step 1)
12. trigger import
13. verify keywords
    ==> NOK (as expected; same results as from step 3)
14. shutdown system (docker-compose down)
15. change PP yml from :latest to :preview (keep other locations like storage/originals/DB)
16. start-up system (docker-compose up)
17. check that system is ":preview" (210425-dd652ce4-Linux-x86_64)
    ==> **OK (**browser informed about upload, hit “reload”)
18. check keywords:
    ==> NOK (kind of as expected; same results as from step 3)
19. trigger re-index of files imported in step 12) (Complete Rescan == true)
    ==> OK
20. verify keywords:
    ==> OK:
    — keyword “flash” is shown (OK)
    — PP generated keywords shown (OK)
    — remaining additional keywords from source file shown as well (OK)
    ====>> Bug fix OK (a re-index of originals seems to take the fix into place for already imported files)

[uLUViL3gCs]

One observation though (irrespective of #1153): For those files of mine, which do have the word "done" (besides others) set as keyword, this word is not shown in PP. Other keywords (next to "done") instead are correctly shown.
Any clue?
Is there probably a blacklist for keywords?

[lastzero]

Yes, PhotoPrism has a stopwords list (like many full-text search engines). Those words were previously shown but were not searchable. Since we're parsing and sorting keywords now in order to merge them, this becomes visible. I understand it can be an issue for certain use cases like when having a "done" tag. We can remove that from the stoplist as a quick fix.

[uLUViL3gCs]

There is no urgent need - at least for me - to fix this right away.
Nonetheless, is there a way to get the full list? Having this, I could circumvent this situation from a different angle.