summeRain v2.0.0
Important
v2.0.0 is an early major release. The upload protocol, browser image
pipeline, schema, compatibility behavior, and operational defaults may still
change frequently. Expect frequent patch releases, review every changelog
before upgrading, and pin an exact version or OCI index digest in production.
Back up MySQL and the complete image volume before the first V2 startup.
This release replaces the V1 upload hot path with a resource-aware V2 pipeline
designed for burst uploads of large images on a 3-core, 4 GB host shared with
other services. It keeps existing V1 images readable while moving new Web
uploads toward deterministic client preprocessing, fixed persisted variants,
durable background publishing, and bounded cleanup.
The V1 comparison baseline for this changelog is tag v0.1.0.
Highlights
- Moves expensive decode, resize, format conversion, and compression work from
the server to the browser for new V2 Web uploads. - Persists deterministic WebP variants instead of generating thumbnails on the
first read, eliminating the V1 double-read/double-write hot path for new
images. - Adds resumable, idempotent upload sessions and durable MySQL-backed publish
jobs, cleanup state, storage lineage, and transactional outbox delivery. - Preserves V1 image serving and provides an explicit server-advertised V1
upload fallback for deployments that disable V2. - Adds a bounded deployment profile, disk-pressure admission, finite database
and Redis pools, and incremental cleanup for constrained shared hosts. - Publishes reproducible multi-platform images to Docker Hub and GHCR with a
recoverable, immutable release-tag workflow.
Full Changelog from V1
Browser-side image processing
- Added V2 preprocessing for static JPEG, PNG, BMP, WebP, and AVIF inputs.
- Added signature-based format inspection before decoding, including extension,
MIME, dimensions, empty-file, and animation validation. - Added explicit rejection for animated PNG, WebP, and AVIF files. GIF is not a
V2 upload format. - Enforced a 15 MiB source-file limit and a 50 megapixel source-image limit.
- Added EXIF-aware orientation handling before crop and resize operations.
- Added a dedicated
wasm-vipsworker fast path with HEIF/AVIF decoding,
single-threaded execution, disabled operation caching, and a bounded tracked
cache. - Added a native fallback based on Pica,
ImageBitmap,OffscreenCanvas, and
Canvas 2D. - Added runtime validation that every generated part is a complete WebP
container before it can be uploaded. - Added an eight-minute client-processing timeout, cancellation, and explicit
worker, bitmap, canvas, blob URL, and object URL cleanup.
V2 generates the following deterministic assets:
| Asset | Geometry | Quality | Lifecycle and use |
|---|---|---|---|
master |
Original oriented dimensions | 80 | Persisted full-resolution WebP; owner/admin access |
gallery |
400x400 cover crop | 60 | Persisted My Images and dashboard preview |
admin |
120x160 cover crop | 60 | Persisted Image Management preview, displayed at 60x80 for 2x density |
publish_source |
Aspect-preserving, longest edge at most 2048 | 80 | Temporary upload part used to create publish |
publish |
Derived from publish_source |
Server output | Persisted public/private publishing asset with optional watermark |
- Added SHA-256, byte size, dimensions, quality, processor version, and recipe
version metadata to upload manifests. - V2 does not preserve the original encoded source bytes.
masteris the
full-resolution, quality-80 WebP replacement. - All persisted V2 variants are WebP. AVIF is supported as an input format only.
Resumable upload protocol and browser recovery
- Added the session-based
/api/v1/uploadsAPI for recipe discovery,
initialization, part upload, completion, single and batch status, and
cancellation. - Added required
Idempotency-Keyhandling. The same key and manifest return
the existing session; a different manifest is rejected. - Added durable session states for initialization, transfer, processing,
completion, failure, cancellation, and cleanup pending. - Added all-or-nothing batch status for up to 100 IDs without leaking whether a
missing ID belongs to another user. - Added four-part resumable manifests with strict client/server manifest
matching. - Added byte-accurate XHR progress, two-minute part timeouts, exponential
backoff with jitter, and up to five part attempts. - Added CSRF refresh and replay only for requests explicitly marked idempotent.
- Added best-effort server-session cancellation after browser cancellation or a
terminal part failure. - Added completion recovery that checks durable server state before retrying an
ambiguous completion response. - Added retry classification that can continue polling, reuse the current
attempt, or create a new attempt after terminal or expired state. - Added adaptive publish polling: 2 seconds for the first 30 seconds, 5 seconds
until 2 minutes, and 10 seconds afterward, with a 10-minute deadline. - Browser processing is limited to one image at a time, upload pipelines to
two, and active upload sessions to four. - Part transfer starts at concurrency two and may increase to three on capable,
fast, non-metered connections. - Upload visibility is fixed for the lifetime of an attempt and cannot be
changed while the batch is active. - The client reads
/api/v1/uploads/recipebefore preprocessing. It uses the V1
multipart path only when the server explicitly advertises
v2_enabled=false. - There is no per-file raw-source fallback while V2 is enabled. Browsers that
cannot use either the WASM or bounded native path receive a clear error.
Backend upload validation and immutable storage
- Streams each part directly to staging while computing SHA-256 and validating
byte length, RIFF/WebP structure, dimensions, and animation in one pass. - Rejects truncated containers, forged RIFF sizes, trailing bytes, unexpected
geometry, hash mismatch, invalid MIME/recipe fields, and animated WebP parts. - Avoids loading complete V2 parts into Go memory and avoids a second staging
read solely for format validation. - Promotes
master,gallery, andadmininto immutable content-addressed
storage and verifies the identity of an already existing target before reuse. - Keeps expensive target prevalidation outside the process-level capacity gate
while retaining serialized transaction and promotion control. - Makes completion idempotent and commits the image, fixed variants, quota
accounting, publish job, and processing event as one durable transition. - Deletes
publish_sourceand the upload staging directory after successful
publish. Upload intermediates are not retained as user-visible assets.
Durable publishing and watermarks
- Added asynchronous MySQL-backed publish jobs with priority, retry limits,
leases, renewal, fencing tokens, and expired-worker recovery. - Prevents a worker that lost its lease from committing or recreating a stale
result. - Adds compensating cleanup for exhausted terminal jobs instead of leaving an
image indefinitely half-published. - Snapshots watermark configuration when completion creates the publish job, so
later configuration changes do not alter an already queued image. - Applies the server-side watermark only to the final
publishasset. Client
generatedmaster,gallery, andadminremain unwatermarked. - Uses atomic watermark replacement, process-local locking, filesystem
flock,
and a recovery journal. - Allows jobs with the same watermark snapshot to run concurrently; historical
snapshots are serialized while the active SVG is safely swapped and restored. - Defaults V2 publishing and imgproxy to two workers each, with a documented
one-worker fallback for more constrained or contended hosts.
Fixed serving routes and V1 compatibility
- Added fixed V2 routes:
/i/<asset_link>.webp/i/<asset_link>/master.webp/i/<asset_link>/gallery.webp/i/<asset_link>/admin.webp/i/<asset_link>/publish.webp
- Query parameters do not generate additional V2 sizes or formats.
masterandadminrequire owner or administrator authorization. Private
galleryandpublishassets retain normal private-image authorization.- Existing V1 images remain readable through their original short links.
- V1 dynamic format, width, height, and quality parameters remain available,
with dimensions capped at 4096. - Existing no-size WebP files and background AVIF outputs may remain persisted.
Arbitrary V1 transformations now use bounded temporary files and are deleted
after the final waiting response finishes. - Concurrent requests for the same V1 transformation share one imgproxy job.
- V1 dynamic generation is limited to two active transformations and four
queued transformation keys; excess work returns a retryable busy response. - Background V1 AVIF generation is limited to one operation, enforces a response
cap, and uses temporary-file plus atomic-rename persistence. - With V2 enabled, legacy
POST /api/v1/images/returns4262. Setting
V2_UPLOAD_ENABLED=falsere-enables the V1 multipart upload endpoint.
Image UI and administration
- Added separate queue states for local processing, transfer, and server-side
publication. - Replaced raw-source previews with previews from the processed
galleryblob. - Added per-file retry, batch retry, cancellation on page exit, and deterministic
object URL cleanup. - Changed the upload completion action to copy fixed Markdown containing V2 WebP
publish links. - Removed the upload-page combinations for arbitrary URL/Markdown/BBCode/HTML
and original/WebP/AVIF output. - My Images and dashboard previews now use
galleryfor V2 images. - Image Management uses
adminat 60x80 CSS pixels andmasterfor zooming. - V2 details expose fixed Publish, Master, Gallery, and Admin links; V1 details
retain dynamic transformation controls. - Added pipeline-aware URL resolution and safe token query/fragment handling.
- Added immediate display of newly issued token expiration and local cleanup
after revocation. - Added explicit administrator behavior for active, suspended,
pending-deletion, and internally deleting users. - Clarified that R2 setting changes do not move historical files and that a
separate migration tool is required.
Storage consistency, R2 lineage, and cleanup
- Added a transactional outbox for CDN purge and physical local/R2 deletion.
- Records deletion intent in the same MySQL transaction that removes the
authoritative business row. - Rechecks
image_variantsandimage_filesreferences under the shared
storage lock before physical deletion. - Treats R2 404 responses as successful idempotent deletion.
- Creates a durable cleanup intent before a new V1 R2 upload performs its first
remotePUT. - Records the exact storage backend, endpoint, and bucket for new V1 objects.
- Reads, downloads, and deletes against stored lineage instead of whichever R2
target is currently enabled. - Blocks endpoint or bucket changes while records reference another target,
historical records remain unclassified, or remote deletion events are pending. - Validates R2 endpoint and public URL values and rejects embedded credentials,
query strings, fragments, and non-HTTP(S) URLs. - Removes the synchronous
/api/v1/admin/r2/migratepath. Historical migration
is reserved for a separate checkpointed, auditable, rollback-capable tool. - Does not guess or rewrite historical V1 storage targets at startup.
- Adds retryable staging restoration after rollback and incremental recovery of
unreferenced persistent V2 files. - Retains directory cursors and applies entry, time, and batch budgets so large
staging and orphan trees are not rescanned into unbounded memory. - Keeps failed storage-deletion events until they succeed instead of removing
them through generic retention cleanup. - Streams batch-download ZIP files and uses store mode for already compressed
images instead of assembling archives in memory.
CDN, privacy, and visibility
- Added durable Cloudflare purge delivery plus a generic authenticated webhook
fallback, with bounded batches, leases, rate, and timeout. - Leaves purge events pending when no delivery target is configured instead of
falsely reporting success. - Caps public origin caching at 10 minutes so visibility changes converge even
when active purge delivery is unavailable. - Public-to-private V2 changes switch the active origin alias from
<unique_link>to<unique_link>Sand enqueue purge work for old public URLs. - Private-to-public changes remove the private alias, revoke active access
tokens, and purge affected URLs. - Private responses remain
no-storeacross browser, proxy, and surrogate
cache headers. - Serializes access-token replacement on the image row to preserve one active
token per image during concurrent requests.
MySQL schema, recovery, and account deletion
- Added ordered, additive MySQL migrations under a database-scoped advisory
lock and a reserved connection. - Records immutable SHA-256 checksums in
schema_migrationsand fails startup on
unknown, missing, reordered, modified, or unapplied migration history. - Adds V2 pipeline metadata to
imagesplusimage_variants,upload_sessions,
upload_parts,processing_jobs,outbox_events, global capacity state,
storage-reference indexes, retention indexes, and R2 lineage fields. - Existing image rows remain V1 and completed; startup does not generate V2
variants or move historical files. - Separates
pending_deletionfrom the internal fail-closeddeletingphase. - Performs account deletion in bounded resumable phases and waits for active
uploads and publishing to reach safe terminal states. - Durably cleans variants, deduplicated files, local/R2 objects, sessions,
tokens, notifications, audit data, and upload records while recalculating
shared-file references before deletion.
Resource management for shared hosts
The default container budget is:
| Service | CPU | Memory | PID limit |
|---|---|---|---|
| Backend | 0.75 | 640 MiB | 128 |
| MySQL | 0.75 | 1024 MiB | 256 |
| Redis | 0.15 | 192 MiB | 128 |
| imgproxy | 0.70 | 512 MiB | 128 |
- Sets
GOMEMLIMIT=512MiBinside the backend container. - Bounds MySQL to 8 open and 4 idle application connections with a 30-minute
lifetime, and Redis to an 8-connection client pool. - Configures Redis with AOF, a 128 MB dataset ceiling, and
noevictionso
capacity failures are explicit rather than silently deleting control state. - Adds soft and hard disk-pressure admission thresholds at 80% and 90%.
- Includes all declared parts plus a bounded 32 MiB publish allowance in upload
reservations. - Includes active-session reservations in quota checks so parallel sessions
cannot independently spend the same remaining quota. - Serializes global capacity across backend instances through MySQL and protects
the small database pool with a process-local admission gate. - Defaults to 8 concurrent part uploads globally, 4 per user, and 8 active
backend sessions per user. - Adds a 1 MiB limit for ordinary JSON bodies while preserving endpoint-specific
streaming limits for image uploads. - Defines explicit header, read, write, idle, and large-upload timeouts.
- Adds bounded queues, health-check start periods, graceful shutdown, PID limits,
log rotation, andno-new-privilegesdefaults.
Security and browser compatibility
- Enables COOP/COEP cross-origin isolation by default for the large-image WASM
processing path. - Rejects GeeTest v4 at startup and through runtime administration while
isolation is enabled.none, reCAPTCHA, and Turnstile remain supported. - Adds typed, guarded reCAPTCHA and Turnstile integrations with explicit
unavailable-library errors. - Adds authenticated, exact-origin, Fetch-Metadata-aware CSRF refresh with a
bounded overlap of valid tokens for out-of-order browser responses. - Fails closed for unknown and destructive account states across Web and device
authentication. - Rejects absolute stored paths, traversal, escaping symlinks, and non-regular
files. - Requires staging to be a child of persistent storage and applies restricted
local permissions. - Pins browser production output to ES2020 for the observed Chrome 93+ Android
compatibility floor. - Detects capabilities at runtime instead of using a UA allowlist.
- The 50 MP WASM path requires cross-origin isolation,
SharedArrayBuffer,
Worker support, and approximately 4 GB or more reported device memory. - The native fallback is intentionally capped at 8 MP on low-memory devices,
13 MP on typical or unreported-memory devices, and 16 MP on devices reporting
at least 8 GB, with an 8192-pixel maximum dimension.
Deployment, development, dependencies, and CI
- Removes application-image builds from Compose. GitHub Actions is the only
application image builder. - Requires an exact
DOCKER_IMAGEtag or digest for production and documents
pullplusup -d --no-builddeployment. - Moves staging to
/data/images/.stagingon the persistent image volume for
same-filesystem atomic promotion. - Adds a
storage-initservice that creates V2 directories for UID/GID 10001
with restricted permissions. - Keeps MySQL, Redis, and imgproxy private in production; only the backend is
loopback-published. - Adds
scripts/dev-wsl.shfor dependency-only Compose, direct Go/Vite
development, local HTTPS, and same-origin API/image proxying without a local
application image build. - Adds
requirements.lockand cross-checks it against Dockerfile, Compose,
Go, npm, runtime settings, and release policy. - Locks Go 1.26.5, Node.js 24.18.0 LTS, Alpine 3.24.1, MySQL 8.4.10 LTS,
Redis 8.8.0, imgproxy 4.0.11, libvips 8.18.3, Pica 10.0.2, and
wasm-vips 0.0.18. - Pins third-party GitHub Actions to full commit SHAs.
- Builds
linux/amd64andlinux/arm64images in GitHub Actions with provenance,
SBOM, and shared build cache, then pushes to Docker Hub and GHCR. - Ordinary commits publish
edgeandsha-<12-character-commit>. - Stable v2.0.0 publishes
v2.0.0,2.0.0,2.0,2,latest, and the
commit tag. - Protects exact SemVer tags as immutable while keeping
latest, major, minor,
edge, and commit aliases movable. - Adds digest-aware release recovery across registries. Partial publication is
repaired from an existing descriptor digest; conflicting exact digests stop
the release. - Adds bounded retry handling for Docker Hub policy API failures and keeps
ordinary edge publication independent of that management API. - Synchronizes the root README to Docker Hub after successful publishing.
Breaking Changes
- New V2 uploads require browser preprocessing whenever V2 is enabled.
- V2 does not retain original encoded source bytes;
masteris WebP quality 80. - Animated images and GIF uploads are not supported by the V2 Web client.
- V2 exposes only fixed WebP variants. Arbitrary size, quality, and format
transformations remain V1-only. - V2 uses
/i/<asset_link>.webpand fixed variant paths instead of creating new
assets from query parameters. - AVIF is input-only for V2.
- Default cross-origin isolation requires third-party scripts, fonts, and images
to provide compatible CORS or CORP headers. - GeeTest v4 requires disabling isolation, which removes the large-image WASM
path and is not recommended for 50 MP deployments. - Historical R2 migration is no longer an in-process admin action.
- MySQL, Redis, imgproxy, container paths, non-root identity, and required
environment settings changed; do not reuse production configuration without
reviewing the new examples. - Database migrations are additive and have no automatic down migration.
Upgrading from V1 (v0.1.0)
-
Back up MySQL and the complete
image_storagevolume from the same point in
time. -
Merge new values from
backend/.env.exampleinto the existing private
configuration without replacing secrets. -
Set
DOCKER_IMAGE=jaykserks/summerain:2.0.0or use the published OCI
multi-platform index digest. Do not uselatestfor a production upgrade. -
Ensure
V2_STAGING_PATHis a child ofSTORAGE_PATH; the default is
/data/images/.staging. -
Confirm free disk remains below the 80% soft threshold and files are readable
by UID/GID 10001. -
Review MySQL 8.4, Redis 8.8, and imgproxy 4 compatibility before reusing
existing service volumes. Keep service-level backups. -
Deploy with:
docker compose --env-file backend/.env -f backend/docker-compose.deploy.yml pull docker compose --env-file backend/.env -f backend/docker-compose.deploy.yml up -d --no-build
-
Watch startup for migration checksum and configuration errors, then verify
/health,/ready, V1 image access, a V2 upload, fixed variants, visibility
changes, and CDN behavior.
Existing V1 records and dynamic routes remain supported. The main service does
not bulk-convert historical images. Keep the current historical storage target
available until the separate migration tool is released and completes its own
checkpoint, audit, and rollback process.
Rollback
- Change
DOCKER_IMAGEto the previous immutable tag or OCI index digest and
redeploy with--no-build. - Prefer an application-only rollback. Do not downgrade MySQL or Redis in place
after a newer service version has opened their data volumes; restore a
compatible backup when dependency downgrade is required. - Leave additive V2 migrations and their ledger intact during an application
rollback. - A complete state rollback requires MySQL and image storage from the same
backup point. - V1 does not understand every V2-created variant, job, or lineage record.
Validate images uploaded after the upgrade before treating a V1 rollback as
complete.
Known Limitations
- Animated-image upload is deferred to a later release.
- Historical image conversion and storage migration are not included in the
main service or this release. - V2 persistence currently uses fixed WebP assets on local storage. R2 lineage
improvements in this release primarily protect the V1 compatibility path. - The backend validates container integrity, hashes, geometry, MIME, and recipe
declarations, but does not re-encode client output or independently verify the
visual crop and effective encoder quality. - Large-image support depends on browser and device capability. A browser may
support the input format but still be unable to process it within the bounded
native fallback. - Cross-origin isolation may block third-party resources without suitable CORS
or CORP headers. - CDN purge delivery requires Cloudflare or webhook configuration. Without it,
events remain pending and the 10-minute cache ceiling is the worst-case
convergence bound. - The 3-core/4 GB profile is a conservative baseline, not a universal throughput
guarantee. Disk speed, watermark complexity, database latency, CDN behavior,
and colocated workloads affect the 10-minute target. - Publication is asynchronous. Clients must poll durable upload status rather
than assumepublishexists when the fourth part finishes uploading. - Because this is an early major release, patch releases may be frequent. Pin
exact tags, read each changelog, and avoid moving aliases in production.
Validation
- Backend build, vet, module verification, full tests, and full race tests pass.
- Frontend lint, production build, and 94 automated tests pass.
- Docker/Compose configuration checks, dependency-lock verification, release
recovery tests, Docker Hub policy payload tests, workflow YAML parsing, and
shell syntax checks pass. - The WSL development stack was verified with healthy MySQL, Redis, imgproxy,
backend/healthand/ready, frontend HTTPS, same-origin API proxying, and
COOP/COEP headers.
Compare
Full source comparison: v0.1.0...v2.0.0