[WIP] Gallery folders support

[bnkai]

Add support for gallery folders.
Directories that have a minimum number of images ( set as an option ) inside them (depth 1) are treated during scan as galleries.If a directory has subdirectories that have images inside then each subdirectory will be a gallery.

Auto association of folder galleries to scenes is available.
If /path/to/media/scene.mp4 is a scene (where mp4 can be any of the supported video extensions)
then if /path/to/media/scenefolder exists and has images inside it will be autoassigned to the scene

For checksum the XOR of the MD5s of every folder image is calculated. This has the advantage (over calculating MD5 of all images ) that order doesn't matter so renaming an image inside the folder (and thus changing the read order) doesn't change the checksum.

[bnkai]

Added the folder as gallery classification depending on number of images.

Setting the option to 1, any folder with an image inside is added as a gallery. Sometimes a scene folder has a couple of images inside ( front, back, scene ) and we wouldn't want to add that folder. Setting the option to 4 or 5 for example is useful for those cases.

[bnkai]

A few notes about the limitations of this implementation:
The easiest way ( less changes/no migration needed) to provide support for folders was to maintain the same functions and procedures as the zip files.
That means when a folder is added to the library as a gallery it needs to be treated as a zip file. If you add or remove an image from the folder, without changing the name of the folder, it would have some undesired effects eg the cache probably won't work correctly giving out of order thumbnails while the actual image will probably be ok.
It is advised when editing a folder to rename it so that you can run the clean task and thus get rid of the old checksum and cache. After the clean is complete you can rename it back and rescan. While not as simple as i would want the same holds true for the zip galleries and the generated metadata so i think it's acceptable at least for now.

[joyov]

It is advised when editing a folder to rename it so that you can run the clean task and thus get rid of the old checksum and cache.

When a file is created, deleted or changed, the modify and change timestamp of the directory is updated too. Can we use this to detect changed galleries and automatically remove associated cache entries while library scan is running?

[bnkai]

The modified folder date could work if it's the same across every os but it doesn't change the fact that the folder is now different. So it would need a new checksum also. The rename, clean , scan makes sure that cache and checksum is valid. The modification date also needs DB schema changes as that date would need to be saved somewhere during scan and that's what I wanted to avoid from the start.

[GlassedSilver]

The modified folder date could work if it's the same across every os but it doesn't change the fact that the folder is now different. So it would need a new checksum also. The rename, clean , scan makes sure that cache and checksum is valid. The modification date also needs DB schema changes as that date would need to be saved somewhere during scan and that's what I wanted to avoid from the start.

Just wanted to chime in that whilst I usually love simple code when possible, this ain't it chief.

This is UX-wise very convoluted, unintuitive and to be frank, you're out-dancing the "good" (hated by every user) old Plex dance here, since latter is used to fix specific issues.

Just adding files to a gallery as a reason to redo the gallery... I don't know about that.

Maybe "re-scan all galleries" could be a nice addition? So any known galleries get re-scanned, avoiding going through the entire recursive structure of the library, including all videos?

Somewhere in here we're far off what can be expected by the average user here and quite frankly, even those who "RTFM"ed.

I still appreciate the work done here and that some folks are even this passionate about a medium that is usually seen as disposable.

[bnkai]

I think you are missing the point here.
This is supposed to be a short term solution so that images in folders can get the same functionality as zip gallery files have now in stash.It's a means to get limited support for image folders now.
It is not supposed to be permanent and it will be revisited once we decide what to do with the galleries.( As you may have noticed there are no titles, tags, edit page ,not even a delete gallery button yet )
When that is decided , adding all necessary fields to the db for housekeeping will make it possible to check and update the gallery cache during scan and only where needed.

[echo6ix]

Does this support the WebP image format? I'm finding more and more images using this format.

[bnkai]

It supports everything that is already supported inside zip files , so it should, yes.

[GlassedSilver]

I think you are missing the point here.
This is supposed to be a short term solution so that images in folders can get the same functionality as zip gallery files have now in stash.It's a means to get limited support for image folders now.
It is not supposed to be permanent and it will be revisited once we decide what to do with the galleries.( As you may have noticed there are no titles, tags, edit page ,not even a delete gallery button yet )
When that is decided , adding all necessary fields to the db for housekeeping will make it possible to check and update the gallery cache during scan and only where needed.

That reply really cleared it up for me, thanks so much! :)

[bnkai]

@WithoutPants
i have made some more progress and merged a more recent dev builld in this branch https://github.com/bnkai/stash/tree/gallery_folders_wip but the functionality was left broken. I f you want to take over for a 0.3 merge i advice using this PR's branch as a base and just look at the non functional branch for some extra utility functions/code.

The main issue left for this PR to work properly was how to make sure that the cache is valid and how to update/recreate when the gallery is changed.

I thought of adding 3 more fields in the gallery model , size, image_count, mod_date.

When scanning a gallery the mod_date of the zip file or the gallery folder should be saved in the database. If the db mod_date is different than the one found during the scan then we can have a look at the size. If the size (for gallery folders it could be the size of all images )is also different then the gallery must be updated/changed so the cache folder of the gallery should be deleted and recreated. Size calculation is relatively fast as it can be retrieved along with the mod_time when stating. ( i have a file/utils.go GetFileSizeTime function in the non working branch). A count galleries size could also be calculated from the db and added to the library total.

[WithoutPants]

I've spent a bit of time thinking about this, #24 and #290, and I think that this is the wrong approach that will end up tying us up.

I think we should move towards Galleries being more of a grouping of Images - like playlist or collection for scenes - with the view toward having images as first-class objects. This will make Galleries more like Performers/Studios/Movies in that the user can add and remove galleries without worrying about the underlying files.

We will need to maintain backwards-compatibility with existing zipped-style galleries.

My concern with this approach is that once it is implemented, it will be very difficult to walk it back when we move towards the more dynamic paradigm.

[bnkai]

I have very few image galleries so i can't speak in general. The folders solution seemed good enough for me for 2 main reasons.

1.I wouldn't have to add every single image i had to the DB, just the folders containing them.
2.I wouldn't need to manually create galleries from images i already had categorized in folders.

This ofc has the limitations you mention so it was sort of a temporary solution and thats why i tried to keep things as compatible as possible to the zip galleries. If the migration to a better approach is easier if we skip this PR i don't mind. I would remove the 0.3 milestone though.

[bnkai]

Closing this. Switching to #24 instead