Way to check when image generation is complete

[mikaraunio]

I'm very thankful that django-pictures supports async image processing with multiple MQs out of the box.

When using async processing, it would be useful to have a way of knowing when all resolutions have been processed – for example for displaying a "please wait while your image is processed" overlay.

Thinking of either a boolean "processing_complete" flag field (could be specified like width_field is now) which we could poll, and/or being able to pass a callback to the processing task, which could, say, send a WebSocket message.

[codingjoe]

Hi @mikaraunio,

Before we get into the weeds, thank you for your kind words. They mean a lot <3

Now, let's talk about features: I understand the use case. There really isn't a good way to predict when an image has been fully processed. It would be easy enough to implement some kind of DB field.

However, this will only be applicable, if you use the built-in image processor. If you service images via a CDN with custom image processing, this isn't really helpful. Then again, this package aims to be convenient out of the box for most simple use cases.

Would you care to draft a proposal, on how this should work for a developer? I usually like to start with the documentation, before diving into the implementation.

Cheers!
Joe

[mikaraunio]

Thank you for considering this.

Sorry for not getting back sooner. I ended up going with an image-processing CDN for generating renditions on the project I'm currently working on.

I appreciate that if images are uploaded to S3 or a CDN, there's no way to really know when they will be ready to serve the image. The best we can do is signal when we are done with it, and I think that's still useful info.

Furthermore, when the django-pictures task is done, I believe the upload to S3 / wherever should be complete, and by that point it should be safe for the browser to request the rendition URLs: they should eventually return the image, even if the requests stalls for a bit while the CDN fetches the image from S3 or does whatever it's supposed to do. And would think "safe to request the renditions URLs without getting a 404" is what really matters here.

For the actual implementation, how would you feel about using Django signals? Thinking sending a renditions_done signal might be an easy way to decouple django-pictures and the MQ from the callback, and allow the developer to easily implement whatever logic they wish.

[codingjoe]

Yes, for most serious cases, creating renditions just in time + caching is probably the best solution. However, this does require the setup of a CDN. I am not 100% sure, how to evolve Django pictures to do both: Be an easy off the shelf entry solution and a solid long-term solution to serve trillions of images. That's something that I wanted to address in the next major release. Thanks for your input, I will contemplate it a little until I find time to tackle a new big release.