How to config? How to reclaim space?

[tavinus]

Hi!

Say I apply these settings

sudo -u www-data php /var/www/nextcloud/occ config:app:set --value="64 128" previewgenerator squareSizes
sudo -u www-data php /var/www/nextcloud/occ config:app:set --value="64 128" previewgenerator widthSizes
sudo -u www-data php /var/www/nextcloud/occ config:app:set --value="64 128" previewgenerator heightSizes

And my config.php has

  'preview_max_x' => 2048,
  'preview_max_y' => 2048,
  'preview_max_scale_factor' => 2,

OR

  'preview_max_x' => null,
  'preview_max_y' => null,
  'preview_max_scale_factor' => 2,

And then I run generate-all

Would the generator set the '_max' file to 128 pixels only and then just use that size as maximum even when I click to open an image? Or is just for pre-generating and nothing else and then the bigger image will be generated when clicked? I am but confused because I got some mixed results here, but I am guessing the browser cache was doing its thing.

What would be the recommended settings for a big install with lots of images?
We want to be able to browse it fast and have the best full-screen experience, but we would like to avoid having 30+Gb (still processing) of preview images data (which also goes to the backup, btw).

And then there is also the question if it is safe to remove the preview files and/or folders to reclaim that space, which I already asked on core. nextcloud/server#5263 (comment)

I would probably prefer to make some simple bash scripting to remove just some bigger sized PNGs so I don't have to reprocess everything. That is easy if there is no problem with the database and other stuff.

In any case, because I was testing different configs as well, I can see many different preview images above 1024px for some files, which is probably the reason for such a big footprint.

I would prefer to have just the smaller sizes pre-generated (for menus and icons) and let the big stuff be done as needed. Can this be done with the config above?

Edit

Also, what does pre-generate really do? It runs in 1s and does not seem to create any files here. generate-all scans everything (for days) and sucks the processor and creates the pngs, but pre-generate does not seem to do anything.

[tavinus]

First of all, the most annoying thing that was happening here was that I had max_x/y set to 512 at some point and because of that many of my previews were still opening in 512 pixels even though I had it set to 2048 and null for a long time at my config. I really wanted to reset everything so I could have the bigger previews for those.

The other problem was that my data folder doubled in size, as mentioned (even though I was browsing in 512px).

To "reset" the preview cache

1. Stop Webserver (apache / nginx)
2. Moved the appdata_instaceName/preview folder outside the appdata folder. You can just remove it if you are confident.
3. Run sudo -u www-data /usr/bin/php /var/www/nextcloud/occ files:scan-app-data to fix database
4. Start Webserver and Test everything works
5. Finish up (delete moved folder, etc).

A command to do this would still be nice and needed.
Also something that checks for orphaned previews (from deleted files / removed external storage).
Deletion seems to trigger preview deletion in recent versions but external storage removal does not (I tried).

I am still not sure on the optimal way to use the generator though. I was checking the image sizes generated when browsing and previewing some of my folders and it seems like the 32x32 and 256x256 are the most used for a desktop interface. So I tried using that for squared thumbs and null for the other two but I still got some extra thumbnail sizes generated. Maybe I will try to hack it to those 2 sizes only later.

At this point I gave up on pre-generating. I just browsed the main folders from gallery a little bit so it would pre-generate that and have a fast base tree.

I wonder if we could pre-generate exactly the same sizes needed for the browser on list / thumbs views. Maybe even use the same functions from there? Not sure...

Cheers!
Gus

[nachoparker]

You might be interested in this

https://ownyourbits.com/2019/06/29/understanding-and-improving-nextcloud-previews/

[ghost]

Is there any work done to keep preview files from entering oc_filecache table in the SQL db? Ideally, if they indeed are needed, perhaps a second table or second db for previews would be preferred. It really slows sql performance down a lot.

[ghost]

I think this is an related issue: #162 (comment)

[dfischer02]

First of all, the most annoying thing that was happening here was that I had max_x/y set to 512 at some point and because of that many of my previews were still opening in 512 pixels even though I had it set to 2048 and null for a long time at my config. I really wanted to reset everything so I could have the bigger previews for those.

The other problem was that my data folder doubled in size, as mentioned (even though I was browsing in 512px).

To "reset" the preview cache

1. Stop Webserver (apache / nginx)
2. Moved the appdata_instaceName/preview folder outside the appdata folder. You can just remove it if you are confident.
3. Run sudo -u www-data /usr/bin/php /var/www/nextcloud/occ files:scan-app-data to fix database
4. Start Webserver and Test everything works
5. Finish up (delete moved folder, etc).

A command to do this would still be nice and needed. Also something that checks for orphaned previews (from deleted files / removed external storage). Deletion seems to trigger preview deletion in recent versions but external storage removal does not (I tried).

I am still not sure on the optimal way to use the generator though. I was checking the image sizes generated when browsing and previewing some of my folders and it seems like the 32x32 and 256x256 are the most used for a desktop interface. So I tried using that for squared thumbs and null for the other two but I still got some extra thumbnail sizes generated. Maybe I will try to hack it to those 2 sizes only later.

At this point I gave up on pre-generating. I just browsed the main folders from gallery a little bit so it would pre-generate that and have a fast base tree.

I wonder if we could pre-generate exactly the same sizes needed for the browser on list / thumbs views. Maybe even use the same functions from there? Not sure...

Cheers! Gus

Do not delete the preview folder, only the contents of the preview folder.
Otherwise:

1. cd into the app data folder
2. mkdir preview
3. sudo chown www-data:www-data preview/
4. Continue as described with the files scan-app-data