Import Captures from S3 Data Storage

[mihow]

• New model for configuring S3 Storage Sources, each with their own credentials, custom endpoint, public URL base, bucket name, etc. As well as cached stats about the total file size and number of files.
• New fields on Deployment model for configuring the storage config model, storage subdirectory or regex pattern of files to be included under one deployment (e.g. "/vermont/luna/.*-snapshots.jpg")
• Method to import all source images (captures) in a deployment's storage
• Methods to read and import paginated batches from S3 increase performance of large imports (10k+ files per deployment)
• Expose captures to label studio endpoint
• Add custom pagination for label studio endpoints
• Method to split captures into session events based on timestamp gaps
• Update admin views with helpful list views, filters & actions for managing source images
• First background tasks configured using Celery for importing source images & counting files
• First Django tests that are specific to this project
• Add additional DB indexes & unique_together constraints to SourceImage and Event
• Initial method of formulating public URLs for source images in the object store. This will be reworked once we start resizing images & making thumbnails.

TODO

• Merge migrations
• Add Minio or another way to test S3 safely
• Mature how public URLs are handled for source images & resized images

[netlify]

✅ Deploy Preview for ami-web canceled.

Name	Link
🔨 Latest commit	23111e6
🔍 Latest deploy log	https://app.netlify.com/sites/ami-web/deploys/64e7200a241ca90007b4344a

[netlify]

✅ Deploy Preview for ami-storybook canceled.

Name	Link
🔨 Latest commit	23111e6
🔍 Latest deploy log	https://app.netlify.com/sites/ami-storybook/deploys/64e7200a708e88000765591c