[Feature Request] Centralise thumbnail store and make the location configurable

[mmattel]

Thumbnails are currently created and stored for each user
This has some negative impacts like:

• very high space consumption due to duplicates
• deleting a mount creates orphan thumbs and database entries

Example 20k pictures can create up to 3GB of thumbs.
Sharing the picture store with 100 people creates 300GB of space consumption
If you remove the picture mount, you have 300GB filespace as file and database orphans

This FR is about to centralize the thumbnail store and make the location configurable

Existing issues/Feature Requests I found related were:

#25770 [Feature Request] adding column "ref_storage" in table oc_filecache
#25213 make thumbnail, trashbin and version paths configurable
#22058 (comment) Primary storage - object store and encryption

Note: could be done in steps.

• Step one is deleting all existing user thumbs including db cleanup to eliminate existing orphans and go eg with idea [Feature Request] adding column "ref_storage" in table oc_filecache #25770 what would eliminate the orphan topic generally.
• Step two is centralisation with an "repair" procedure deleting all post created user thumbs and setup the central thumb store

[PVince81]

Had a discussion with @butonic for adding a new "target_fileid" column in oc_filecache for future symlink support and maybe pointing directly to target storages for mounts. If we had such column, it could also be used to achieve the purpose of the "ref_storage" you suggested.

For encryption, I think currently thumbnails aren't encrypted. If one day we encrypt them, then they'd need to be encrypted with the same keys like the original file, share keys, etc, so everyone who has access to the original file also has access to the thumbnail. It might need additional hook listeners in the thumbnail code.

[PVince81]

@butonic @DeepDiver1975

[mmattel]

Another use case beside thumbs for pictures would be thumbs for/from audio/video files (eg cover or album art)

[PVince81]

There was an older ticket about this feature request: #17186

[PVince81]

linking another ticket with more discussions #10808

[PVince81]

another one. #17186

[PVince81]

Linking to duplicate: #19548

[mmattel]

As user_x which has admin status, I had historically all my private mounts under admin and not under personal settings. I corrected this today by removing all mounts affected from admin and readded them under personal. Because I also have many pictures, I was wondering about the quantity of items rescanned.

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 14117   | 93859 | 00:08:16     |
+---------+-------+--------------+

Remembering about this issue here, I deleted manually the orphaned thumbs for user_x and rerun scanning...

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 3792    | 66132 | 00:05:36     |
+---------+-------+--------------+

The difference was 4.2G in:

+----------+-------+
| Folders  | Files |
+----------+-------+
| 10325    | 27727 |
+----------+-------+

Thumbnail orphans is really an issue !

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.