Skip to content

feat: add S3 storage, locking, concatenation, and JSON metadata serialization - #111

Open
tomdesair wants to merge 13 commits into
masterfrom
feature/s3-storage-support
Open

feat: add S3 storage, locking, concatenation, and JSON metadata serialization#111
tomdesair wants to merge 13 commits into
masterfrom
feature/s3-storage-support

Conversation

@tomdesair

Copy link
Copy Markdown
Owner

Summary

This pull request introduces full S3-compatible object storage support (S3StorageService), distributed locking (S3LockingService), dual-strategy concatenation (S3ConcatenationService), optional JSON metadata serialization, and developer instructions for Testcontainers + MinIO (supporting Docker and Podman).

Key Highlights

  • S3StorageService: Always-multipart upload strategy matching tusd, incomplete part buffering (.part objects for sub-5MB chunks), dynamic part scaling up to 5GB, checksum deduplication, and zero-byte upload support.
  • S3LockingService: S3 conditional writes (If-None-Match: "*"), heartbeat daemon lease renewals, cross-pod lock contention resolution (.stop signal files), enabling multi-replica container deployments without external Redis/database lockers.
  • S3ConcatenationService: Fast server-side copy (UploadPartCopy) for partial uploads >= 5 MB and streaming re-upload fallback for sub-5MB chunks.
  • JSON Serialization: Added .withJsonSerialization() builder option for TusFileUploadService and DiskStorageService.
  • Integration Tests & Developer Docs: Added ITS3StorageService, ITS3LockingService, ITS3TusFileUploadService, and ITS3RufhProtocol integration test suites running against MinIO using TestUtils (supporting Docker & Podman). Added developer documentation in docs/S3_STORAGE.md.

Verification

  • All 47 integration test scenarios pass on MinIO (0 skipped).
  • 100% JaCoCo code coverage on new and modified lines.

@coveralls

coveralls commented Aug 2, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 95.854% (-0.6%) from 96.487% — feature/s3-storage-support into master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants