Implement Cantaloupe for thumbs delivery channel ingest

[donaldgray]

EDIT 6 Feb 2024

This task is to implement the ADR from #722

---

Updated 20 Sept 2023:

Instead of Appetiser using Pillow to make the simple, square-confined thumbnails from config like "1024,400,200,10"...

...it (or some other step of the ingest, driven by Engine) invokes its own personal sidecar Cantaloupe, using the configured sizes in the delivery channel policy to request all the required thumbs and save them to S3 with an updated s.json.

This cantaloupe has its own scratch volume. The engine process copies the Jp2 to the volume, requests all the sizes, saves the details to S3 in exactly the same format as current thumbs output, and then deletes the JP2 from its scratch volume.

There's no need for any scavenging, even if the occasional JP2 gets left behind due to an error, the sidecar instance can just be disposed of, its storage is completely ephemeral, nothing else depends on it.

This process will need to load the bitmaps it gets back from Cantaloupe to determine their actual height and width.
It doesn't need to calculate them in anticipation of what Cantaloupe might return. The stored sizes in s.json are therefore exact - measured not computed.

As noted in the docs, if the image is too small for Cantaloupe to return a valid response for a particular size, this is not counted as an error or recorded as one (might be simply logged). It just omits that size from the output s.json.

This task includes refactoring Engine to call this service, after Appetiser (and if it's already use-original, not even calling Appetiser).

(original comments below preserved)

---

original Description, Mar 1 2022:

Ticket based on #30 (comment)

Could Engine use SpecialServer to make thumbs? Instead of or as well as Appetiser?
And could it do them on demand, after the fact - e.g., if you added some extra sizes to the thumbnail policy for an image. The Thumbs service, just before it returns a 404 for a non existent thumb size, could check the policy and see if it is actually a supported size and create a new thumb on demand - either by reducing the next thumb size up using the above imageflow, or if the new size is bigger than any existing thumbs, by invoking imageflow on the source image?

NB imageflow does NOT support JP2 and has no plans to.
But SpecialServer might have its own personal Cantaloupe / IIP to call on, as well, maybe...

[tomcrane]

This should be done with #66 - then the ingest process (Appetiser or its successor) is just passing through Image API params:

for each entry in policy,
   request /full/{entry}/0/default.jpg
   save to S3
   add to s.json array (measure the image or calculate what's expected?

[donaldgray]

A thought on above - whichever deployment of Cantaloupe we use will need to have access to the source image / generated derivative in order to create thumbnails. The simplest approach may be for Engine to have it's own Cantaloupe, although this may not be feasible due to the constrained size of the Engine tasks.

For SpecialServer to be used the Engine flow could become complex as the full "transcode and upload" process would need to be complete prior to thumbs being generated.

[tomcrane]

This fits around issues #626, #627, #629

[tomcrane]

Need to answer whether we can run cantaloupe as a sidecar to Engine, or whether we use SpecialServer, passing it the temporary S3 location. SPIKE => RFC.

[tomcrane]

To be refined -

What is the relationship of this task to #629?

This task is infra for Cantaloupe acting as thumb generator?

[tomcrane]

Closing as it's now just infra, so is contained within #629