Skip to content

v1.10.0

Choose a tag to compare

@mstgnz mstgnz released this 03 Aug 10:42
· 11 commits to main since this release

Cold storage archiving, a stored-XSS fix, and the end of the daily OOM restarts.

Full detail in CHANGELOG.md.
Upgrade checklist in docs/migration.md.

⚠️ Action required before upgrading

  • MinIO (9000, 9001) and Redis (6379) are now bound to 127.0.0.1 instead of every interface. If anything outside the host reached them directly it will stop working. That is the point: a published Docker port bypasses ufw, so these were previously on the public internet of any host with a public IP. Override with MINIO_BIND / REDIS_BIND if you genuinely need the old behaviour, and rotate MINIO_ROOT_PASSWORD if they were ever reachable.
  • Remove Cache-Control from proxy_ignore_headers in any proxy in front of this service. The service now marks a degraded resize response no-store, and a proxy that ignores it will cache a full-size image under a ?width= URL.
  • aws_upload no longer does anything. Archiving is a property of the deployment, not of the request. The field is still accepted so existing clients do not fail validation.
  • Choose ARCHIVE_BUCKET before the first backfill. The mapping is resolved on every request, so changing it later strands everything already archived.

Security

  • Stored XSS through text uploads (.csv, .sql). Content validation accepts valid UTF-8 text, and DetectContentType calls anything opening with markup text/html. Together they meant <script>…</script> uploaded as evil.csv was served as HTML on this service's own origin. nosniff was no defence: the browser was not guessing, it was being told. Sniffed HTML and XML now go out as text/plain, bytes unchanged. Present since .sql was allowlisted.
  • MinIO and Redis are no longer published on 0.0.0.0 (see above).

Added

  • Cold storage archive. Uploads are mirrored to S3 in the Glacier Instant Retrieval class when AWS credentials are configured, and objects MinIO no longer holds are served from there at the same URL. Instant Retrieval is what makes that possible: the older Glacier classes need a restore request and hours of waiting.
  • POST /archive so the application that owns the content decides when an object stops needing to be local. Accepts object keys or full CDN URLs, evict: false to copy without freeing, idempotent so batches are safe to retry.
  • Retention job that deletes local copies of objects the archive is confirmed to hold at a matching size. Off by default, reporting-only the first time it is enabled.
  • backfill and restore commands, shipped in the image. backfill copies what predates the archive into it; restore is its inverse, so adopting cold storage is not a one-way door. Neither deletes anything.
  • ARCHIVE_ONLY_BUCKETS to narrow which buckets are archived. Gates writing, never reading, so narrowing it cannot break URLs.
  • .doc, .docx, .csv and .zip uploads.
  • mc client container behind the tools compose profile.

Fixed

  • Replicas were OOM-killed roughly seven times a day. ImageMagick starts one OpenMP thread per core and allocates through malloc, so glibc keeps a per-thread arena it never returns to the OS. Capping threads (IMAGICK_THREAD_LIMIT=2) and arenas (MALLOC_ARENA_MAX=2) took a production deployment from 2.9 GB to roughly 200 MB per replica.
  • The GET resize path had no concurrency bound at all, so one gallery page started twenty simultaneous full decodes. Now RESIZE_MAX_CONCURRENT.
  • Archived objects were empty. S3PutObject was handed the reader the MinIO upload had just drained, so every archived object was zero bytes. Nothing surfaced it because the upload still reported success.
  • Every cache miss was logged as an error, which on the rate limiter meant one ERROR line per request.
  • nginx asked the upstream to upgrade every connection, so the keepalive pool was never used.

Changed

  • The retention sweep runs its per-object work concurrently; serially, a pass over a few million objects took days.
  • nginx.conf no longer carries a response cache. The directives were inert, and activating them meant a deleted object kept being served with no way to purge.