Index: Don't use `*.thm` thumbnail files as primary image

[pgalbavy]

---

1. What is not working as documented?

As per yesterday's conversation, I have many old Canon CRW/THM photo sets. PP is seeing the THM as JPEG as making that primary even though it is only 160x120 in size.

I then added *.thm to .ppignore which correctly skipped the THM files but then when processing the CRW files it added the THM back.

time="2023-11-20T15:30:18Z" level=info msg="index: ignored "2002-01-26 Tate Modern/CRW_9183.THM""
time="2023-11-20T15:30:18Z" level=info msg="index: added main jpg file '2002-01-26 Tate Modern/CRW_9183.THM'"
time="2023-11-20T15:30:18Z" level=info msg="index: stacked related raw file '2002-01-26 Tate Modern/CRW_9183.CRW'"

In subsequent testing I found if I manually created larger JPEGs using photoprism convert -ext crw ... and I renamed the file to have THM instead of CRW, the index picked up the new JPEG:

$ mv CRW_8897.CRW.jpg CRW_8897.THM.jpg

...
INFO[2023-11-20T16:25:52Z] index: updated main jpg file '2002-01-02 Dunstable Downs/CRW_8896.THM'
INFO[2023-11-20T16:25:52Z] index: updated related raw file '2002-01-02 Dunstable Downs/CRW_8896.CRW'
INFO[2023-11-20T16:25:52Z] index: ignored "2002-01-02 Dunstable Downs/CRW_8897.THM"
INFO[2023-11-20T16:25:52Z] index: updated main jpg file '2002-01-02 Dunstable Downs/CRW_8897.THM'
INFO[2023-11-20T16:25:53Z] index: updated related raw file '2002-01-02 Dunstable Downs/CRW_8897.CRW'
INFO[2023-11-20T16:25:53Z] media: created 11 thumbnails for '2002-01-02 Dunstable Downs/CRW_8897.THM.jpg' [804.847715ms]
INFO[2023-11-20T16:25:53Z] index: stacked related jpg file '2002-01-02 Dunstable Downs/CRW_8897.THM.jpg'

2. How can we reproduce it?

1. Add CRW/THM files and index.
2. Review logs, see above behaviour.

3. What behavior do you expect?

I would expect the THM file to be ignored unless "Preview files" is explicitly disabled.

In the second instance I would expect the .ppignore entry to result in the files being ignored for all stages of processing.

4. What could be the cause of your problem?

Quick browse of code shows remnants of THM files not being classes as valid JPEGs but from discussion it may be this code path is no longer used.

5. Can you provide us with example files for testing, error logs, or screenshots?

Sampler files submitted as requested via email.

6. Which software versions do you use?

(a) PhotoPrism Architecture & Build Number: amd64 on docker with :latest

(b) Database Type & Version: MariaDB from docker :latest

(c) Operating System Types & Versions: Ubuntu 22.04 LTS, docker CE 24.0.2

(d) Browser Types & Versions: Chrome / Edge

(e) Ad Blockers, Browser Plugins, and/or Firewall Software? Lots. Not relevant.

7. On what kind of device is PhotoPrism installed?

Generic i7, 8GB, SSD for root and PP storage, btrfs on external spinning rust for originals.

8. Do you use a Reverse Proxy, Firewall, VPN, or CDN?

NGINX with Letsencrypt in another docker container on an RPI4

[graciousgrey]

Thank you for creating the report. I can confirm that. PhotoPrism does not yet allow you to completely ignore certain file types. If you add a file extension to .ppignore, standalone files of that type will be ignored. However, if there are files of a different type with exactly the same path/name, the files with the ignored extension will be stacked with these files. If the THM files were in a different folder, they would not be stacked with the CRW files.

The option to completely ignore certain file types is on our to-do list.

Moreover, we should prevent THM files from being used as primary files so that it is not necessary to ignore them to get better preview images.

[pgalbavy]

For now I have added both CRW and THM to .ppignore and doing a full reinstall and fresh index. I hope that once there is time to address this issue I will be able to simply remove the entries and re-index.

[pgalbavy]

I was going to dive in a little bit further myself after updating to the latest build (today). I built a test environment (on another server, obv) as per https://docs.photoprism.app/developer-guide/setup/ and got it up and running easily.

I then uploaded two pairs of images from the directory I used in my initial report and they imported correctly, no THM file used. I thought to myself perhaps some side effect of other changes, and went to my "production" system and changes .ppignore to no longer ignore "*.thm" and ".crw" and tried a limited rescan of the same directory - nope, same issue as in the report.

What can be different about the docker test environment that means it doesn't show the same issues?

[lastzero]

I don't see how THM files would be identified as JPEG files given the following code:

photoprism/internal/photoprism/mediafile.go

Lines 667 to 680 in bb434f7

	func (m *MediaFile) IsJpeg() bool {
	if m.Extension() == fs.ExtTHM {
	// Ignore .thm files, as some cameras automatically
	// create them as thumbnails.
	return false
	} else if fs.FileType(m.fileName) != fs.ImageJPEG {
	// Files with an incorrect file extension are no longer
	// recognized as JPEG to improve indexing performance.
	return false
	}
	// Check the mime type after other tests have passed to improve performance.
	return m.MimeType() == fs.MimeTypeJPEG
	}

Also see:

photoprism/internal/photoprism/mediafile.go

Lines 661 to 664 in bb434f7

	// IsPreviewImage return true if this media file is a JPEG or PNG image.
	func (m *MediaFile) IsPreviewImage() bool {
	return m.IsJpeg() || m.IsPNG()
	}

However, if you have *.thm.jpg files created with e.g. the photoprism convert command, they would certainly be indexed and the THM files might be visible in the list of sidecar files. The import process could also have created the additional JPEGs when it found the THM files and could not identify them as genuine JPEGs (due to the code shown above).

Having both an image.jpg and an image.thm.jpg file calls for problems, as the base name image (i.e. without all extensions) is absolutely the same and the file type is also the same.

If so, it seems possible to add yet another fix / workaround for this specific case or to ignore THM files completely from the start?

[pgalbavy]

There are no pre-converted "large" jpegs, that was just a specultive workaround/test:

peter@ubuntu:/media/hdd1/Pictures/2002/2002-01-02 Dunstable Downs$ ls
CRW_8876.CRW  CRW_8878.CRW  CRW_8880.CRW  CRW_8882.CRW  CRW_8884.CRW  CRW_8886.CRW  CRW_8888.CRW  CRW_8890.CRW  CRW_8892.CRW  CRW_8894.CRW  CRW_8896.CRW
CRW_8876.THM  CRW_8878.THM  CRW_8880.THM  CRW_8882.THM  CRW_8884.THM  CRW_8886.THM  CRW_8888.THM  CRW_8890.THM  CRW_8892.THM  CRW_8894.THM  CRW_8896.THM
CRW_8877.CRW  CRW_8879.CRW  CRW_8881.CRW  CRW_8883.CRW  CRW_8885.CRW  CRW_8887.CRW  CRW_8889.CRW  CRW_8891.CRW  CRW_8893.CRW  CRW_8895.CRW  CRW_8897.CRW
CRW_8877.THM  CRW_8879.THM  CRW_8881.THM  CRW_8883.THM  CRW_8885.THM  CRW_8887.THM  CRW_8889.THM  CRW_8891.THM  CRW_8893.THM  CRW_8895.THM  CRW_8897.THM

I agree that the code seems to do the right thing, but I still get the same/similar problem as in my original report. But not on the docker dev instance (on manual upload, not scan).

Here is the production server with the ignores removed:

time="2024-04-22T10:44:47Z" level=info msg="indexing files in '2002/2002-01-02 dunstable downs'"
time="2024-04-22T10:44:47Z" level=info msg="classify: loading nasnet"
time="2024-04-22T10:44:51Z" level=info msg="classify: loading labels from labels.txt"
time="2024-04-22T10:44:52Z" level=info msg="index: added main jpg file '2002/2002-01-02 Dunstable Downs/CRW_8876.THM'"
time="2024-04-22T10:44:52Z" level=info msg="index: stacked related raw file '2002/2002-01-02 Dunstable Downs/CRW_8876.CRW'"
time="2024-04-22T10:44:52Z" level=info msg="index: added main jpg file '2002/2002-01-02 Dunstable Downs/CRW_8877.THM'"
time="2024-04-22T10:44:53Z" level=info msg="index: stacked related raw file '2002/2002-01-02 Dunstable Downs/CRW_8877.CRW'"
time="2024-04-22T10:44:53Z" level=info msg="index: added main jpg file '2002/2002-01-02 Dunstable Downs/CRW_8878.THM'"

[lastzero]

Did you also check the sidecar storage folder or just the originals folder?

Please also let us know the exact build number your production instance uses. Thank you!

[pgalbavy]

OK, before I do that I have found that it's for scanning files in originals, not for uploads. For uploads THM files are correctly ignored, but I mounted a local copy of just the above directory in the docker dev environment and get the same, unwanted, behaviour. THM files are used as the master JPEG.

With additional debug from dev image:

INFO[2024-04-22T11:33:24Z] indexing originals...                        
INFO[2024-04-22T11:33:24Z] classify: loading nasnet                     
2024-04-22 11:33:24.212661: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /go/src/github.com/photoprism/photoprism/assets/nasnet
2024-04-22 11:33:24.347779: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { photoprism }
2024-04-22 11:33:24.898271: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2024-04-22 11:33:26.676756: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { photoprism }; Status: success. Took 2464129 microseconds.
INFO[2024-04-22T11:33:27Z] classify: loading labels from labels.txt     
INFO[2024-04-22T11:33:27Z] index: found no .ppignore file               
INFO[2024-04-22T11:33:27Z] folder: added album '2002-01-02 Dunstable Downs' (path:"2002/2002-01-02 Dunstable Downs" public:true) 
INFO[2024-04-22T11:33:27Z] index: added folder /2002/2002-01-02 Dunstable Downs 
DEBU[2024-04-22T11:33:27Z] exiftool: extracting metadata from '2002/2002-01-02 Dunstable Downs/CRW_8876.THM' 
DEBU[2024-04-22T11:33:27Z] exiftool: extracting metadata from '2002/2002-01-02 Dunstable Downs/CRW_8877.THM' 
DEBU[2024-04-22T11:33:27Z] exiftool: extracting metadata from '2002/2002-01-02 Dunstable Downs/CRW_8878.CRW' 
DEBU[2024-04-22T11:33:27Z] cache: created 21e73f87136344e94b6004d57ac48d6422c4e06a_exiftool.json 
DEBU[2024-04-22T11:33:27Z] cache: created 55cb7d14b6fc3092df0c93364c327e4980d73a34_exiftool.json 
DEBU[2024-04-22T11:33:27Z] cache: created 887f1396cfacf7f7f54bdb27fd985632e0f890a6_exiftool.json 
INFO[2024-04-22T11:33:27Z] index: added main jpg file '2002/2002-01-02 Dunstable Downs/CRW_8876.THM' 
DEBU[2024-04-22T11:33:27Z] file fsccb08z3i2sl7jn: renaming 2002/01/20020102_101044_C853905D.crw to '2002/2002-01-02 Dunstable Downs/CRW_8876.CRW' 
INFO[2024-04-22T11:33:27Z] index: added main jpg file '2002/2002-01-02 Dunstable Downs/CRW_8877.THM' 
INFO[2024-04-22T11:33:27Z] files: moved existing sidecar to '2002/2002-01-02 Dunstable Downs/CRW_8876.jpg' 
INFO[2024-04-22T11:33:27Z] files: moved existing sidecar to '2002/2002-01-02 Dunstable Downs/CRW_8876.yml' 
DEBU[2024-04-22T11:33:27Z] file fsccb08n8cmdrhkg: renaming 2002/01/20020102_101050_A298A7C5.crw to '2002/2002-01-02 Dunstable Downs/CRW_8877.CRW' 
DEBU[2024-04-22T11:33:27Z] index: CRW_8876.CRW has new photo uid psccdfr7li8dqjd8 
DEBU[2024-04-22T11:33:27Z] index: CRW_8876.CRW was renamed              
DEBU[2024-04-22T11:33:27Z] metadata: no native file format support for CRW_8876.CRW, performing brute-force exif search 

[pgalbavy]

I would imagine it's because of some decision in the flow that says "if the file is already in originals, it cannot be ignored."

In fact, now I look deeper in my logs, I have also added *.wav to .ppignore and yet it logs errors about unknown file types when scanning. I would guess that .ppignore is itself ignored for scanning existing files in originals?

[lastzero]

Does it have the right permissions, so your instance can access it? You could also try trace log mode to see additional logs:

• https://docs.photoprism.app/getting-started/troubleshooting/docker/#viewing-logs

When you have a chance, please check the sidecar storage folder for any related sidecar files and let us know the exact build number of your production instance. Thank you very much!

[pgalbavy]

It is accessing it fine and there were no sidecars files as I originally cleaned everything out and did a complete reinstall early on.

For the above test it's a clean dev docker environment.

I think my comments about .ppignore are not relevant as it's more likely the tests you quotes code for are not fully used in scanning originals, as .THM files should be skipped completely.

[pgalbavy]

"prod" build is "Build 240420-ef5f14bc4" but I can reproduse the problem in "develop" branch from github with the docker env.

[pgalbavy]

Could it be in here?

photoprism/internal/photoprism/index_mediafile.go

Lines 288 to 297 in bb434f7

	if !file.FilePrimary {
	if photoExists {
	if res := entity.UnscopedDb().Where("photo_id = ? AND file_primary = 1 AND file_type IN (?) AND file_error = ''", photo.ID, media.PreviewExpr).First(&primaryFile); res.Error != nil {
	file.FilePrimary = m.IsPreviewImage()
	}
	} else {
	file.FilePrimary = m.IsPreviewImage()
	}
	}

[lastzero]

If this is not already the case, it seems like a good idea to explicitly ignore THM files in the "convert" logic, so that no JPEG files are created from THM files under any circumstances.