Write RFC for managing Asset/DeliveryChannel metadata

[donaldgray]

Ticket #631 requires accessing thumbnail sizes in bulk. This would involve making bulk requests to S3 to read the data which would be inefficient at scale.

To counter this we need to investigate a way of storing metadata for Asset/DeliveryChannel in the database. Initially this will be storing available thumbnail sizes but could be extended in the future to store:

• Thumbnails: generated sizes - both system and requested thumbs. Open and Auth thumbs are currently in json in S3. Would we want to store in S3 and database, or just database? Store open and auth?
  • Should OrchestrationAsset read from DB rather than S3 (saves an S3 GET request).
• Timebased: Generated transcode types and output locations.
• File: do we store a copy of the original file? If so, where.
• Image: do we store a copy of the file. Is it original (use-original) or a transcode?
• Adjuncts: what is stored where (future).

All of the above can be calculated from ImageDeliveryChannels + Engine processing but this can act as an easy lookup to those values. The above metadata would also help with:

• Cleaning up no-longer used manifests (Delete no-longer required delivery artifacts #430)
• Generating full single-item manifest that contain all available representations of an asset (not just image services - see Use Single Asset Manifest to describe all derivatives and services #488)

Questions:

• Assuming this is in a separate table, what is that called? AssetMetadata or DeliveryChannelMetadata (if we store adjunct data is this classifiable as DC related?
• How is the data modelled - column per data type (File, Image, Thumbs) or a JSON blob?

[donaldgray]

Final RFC - https://github.com/dlcs/protagonist/blob/develop/docs/rfcs/016-asset-metadata.md