Feedback during export

[pedrorrodriguez]

When you export an image, the app provides info on two States: "exporting", and "exported". In lower end computers there is no way of knowing if the pipeline is just heavy, or the computer froze.

There is a progress bar for multiple exports. Synchronizing the progressbar with the pipeline progression would make the export even slower.

Originally posted by @aurelienpierre in #252

To solve:

1. in pixelpipe_hb.c, in dt_dev_pixelpipe_process_rec(), map the pos argument to a completion coeff,

2. the function calls itself recursively from the end of the pipe, with pos = number of modules in pipe (typically, 83), until pos == 0, meaning we load the raw buffer into pipeline,

3. store initial_pos = pos at the first call, then progress is computed from pos / initial_pos when running each module process() or fetching input buffer from cache. But the caching vs. processing anew paradigm makes it hard to tell if, for a given pos, we are currently processing or only calling the module (n - 1)

4. find a way to communicate the current valid pos / initial_pos completion coeff to the GUI thread updating the progress bar. Doing so in a thread-safe way, taking into account that not all exports have a GUI and therefore a GUI progress bar, is going to be unreliable and prone to segfaults with current code.

TL;DR: a serious clean-up is needed before even thinking of implementing that.

Originally posted by @aurelienpierre in #252

The mandatory serious cleanup was completed today.

Originally posted by @aurelienpierre in #252

Not sure if syncing progress bar to pipeline is actually needed. At the moment we get a progress bar that advances every time a picture is exported. This progress bar is continuous and static.

How about:

• The progress bar is a series of segments, one for each picture. If you export 2 images, 2 segments, if you export 30, 30 segments.
• Exported images are marked with a static color segment (this goes away when all pictures are exported, obviously).
• The current image being exported is marked by a blinking color segment.
• Pending images are greyed out, have no segment at all, or something like that.

This way you get clear visual feedback of where the export process is, and the blinking indicator gives you some degree of confidence that the app didn't just freeze on you.

[lu-k]

You are thinking of feedback per image with overlays on their respective thumbnail, right?

To see if an app froze I would be ok with just animation somewhere, like a classic loading spinner. For exports there is already a dedicated status in the bottom of the left panel, so I think having one animated element there, signalling "still working" would be helpful.

However I think this ties in to the greater topic of "notification areas" and "feedback on computation". Where do we display what info?

[pedrorrodriguez]

I'm talking about improving that status indicator. It is static and only changes after an image has been exported. If the 9th segment of that progress bar were to blink that should be enough info that something is happening.

However I think this ties in to the greater topic of "notification areas" and "feedback on computation". Where do we display what info?

Yes

[Gamosphere]

Why do we leave the export window opened upon completion of the export(s?

In cases like that it is pretty to see a 'do you want to close the window when completed' check box.

Food for thought in your conversation.

[pedrorrodriguez]

Never bothered me tbh, but maybe a nice detail

[sahlirc]

I may add that having a feedback on Ansel's working in the background, regardless whether it is a progress bar or a popup/message/spinning wheel, would be useful overall.
One case is thumbnail's updating of many images (applying a style for instance). My only way to know that Ansel is working is by looking at the thumbnails being updated (even working is not always shown, until it reaches the set of displayed thumbnails), and/or by enabling the system monitor to look at the processor load and memory usage. Forget about doing anything with Ansel (one may think that it hangs and may be tempted to force quit, bad idea) until both go down (it may last a few minutes depending on the number of images). Then on quitting Ansel, I do the same if I have enabled disk caching. So that I do not open Ansel too early, i.e. before it has written all the thumbnails to disk, otherwise all the previous thumbnail's update is lost. Obviously this happens only when many images are processed.

[maesejofial]

In addition to what's already been mentioned and the need for better code, this is the result of importing 10,008 .RAF images and other JPG formats into the library (not disk). This might be helpful...

[maesejofial]

Second attempt...Second attempt... Twelve minutes import time for 3276 images out of 10000. Ansel freezes. The interesting thing is that when I open Ansel, I have "almost" all the folders with their images imported into the library. But, as they say, the joy is short-lived. According to my system, the folder for the year 2024 has 169 subfolders. In Ansel, I only have 108. That means I'm missing 11. And now I have to go looking for them one by one.

[sahlirc]

I think you expect too much of Ansel during import. I also experienced this issue, and being impatient I force closed Ansel, which is a bad idea and totally counterproductive (see 7 below). I notice from your screenshot, that the thumbnails of the imported images are being worked out by Ansel during import (so you expect that it creates the thumbnails for your 10'000 images, at one second or a little more per image on my computer, this translates to at least 3 hours!), which is a lot of work during which Ansel seems to hang. Try this, as it was impressively fast in my computer (32 G ram, 12 processors, 6 cores, 2.6 Ghz and ssd drives).

1. very importantly before importing, deselect all filters, hence not a single image will be displayed;
2. import many images (I did with > 30'000 of which cr2, cr3, raf, jpeg and tiff);
3. the images will be imported in the database very quickly, but be patient yet and wait maybe a few minutes;
4. afterwards, do not attempt to display all images at once (better do it by folders(s) containing no more than 1500 images) and if you do, select all filters and set the embedded jpgeg display for unedited only, maybe 12 column-display, and go through the images in the lighttable using page down/up, not mouse scrolling;
5. use the system monitor to assess the workload on the computer, in particular processors and memory usage: you know that the job has been done when the processor load goes back to idle;
6. you can close Ansel at this stage: however very importantly look at the memory usage and do not reopen Ansel to generate thumbnails of the next set of images until the ram goes back to its normal low value;
7. once Ansel is closed, actually the thumbnails loaded in the ram are written on disk. If one reopens Ansel too quickly, all thumbnails not written on disk must be worked out again by Ansel. I did that being too impatient and it drove me nuts because it reworked thumbnails I thought had already been processed, until I realized that Ansel transfers the thumbnails to disk after being closed.
8. you can also select a set of images and run preloading cache rather than going through the collection in the lighttable. A progress bar will be shown at the bottom of the left panel. The thumbnails are written to disk. Actually this maybe the way to go!

Hope this helps.