v0.18.2

Highlights

s2-server keeps its per-bucket state — the generation that binds a multipart upload to the bucket it was started in — in its own .buckets/ directory instead of the .meta/ directory the osfs and memfs backends keep for object sidecars. v0.18.1 reserved .meta on those backends and refused Sub(".meta"), which is right, but left server.Buckets reaching in through fs.SubSidecar, a type assertion on the concrete backend. A storage that wraps an osfs or memfs root — what s2.RegisterNewStorageFunc exists for — did not satisfy it, and every CreateBucket on such a root failed with invalid name: .meta is reserved for object metadata. Buckets now speaks s2.Storage and nothing else.

Changes

Fixed

• Create, list and delete buckets on a storage that wraps an osfs or memfs root, which v0.18.1 refused (#276)

Deprecated

• fs.SubSidecar: nothing outside the backend reaches the sidecar directory any more. It is removed in v1.0.0 (#276)

Upgrading

• s2-server operators running the image: docker pull mojatter/s2-server:0.18.2 (:latest points at it too)
• Submodule consumers: go get github.com/mojatter/s2/<module>@v0.18.2
• cmd/s2-server/v0.18.2 is published too: go install github.com/mojatter/s2/cmd/s2-server@v0.18.2
• No object changes and no configuration changes. What moves is one internal directory.

s2-server operators

• Rename the directory while the server is stopped and nothing is lost: mv "$S2_SERVER_ROOT/.meta" "$S2_SERVER_ROOT/.buckets". The two directories have the same shape, and mv keeps modification times, which is what a generation is — so every bucket keeps the generation it had and multipart uploads in flight across the upgrade complete normally. Do it before the new version starts for the first time, and only while .buckets/ does not exist yet: once it does, mv moves .meta inside it rather than renaming it, and the markers end up at .buckets/.meta/ where they do nothing — silently, unless that directory is already populated. s2 cannot do this for you: it addresses storage by object name, writing a marker at the new path would stamp it with the time of the write, and .meta is a name the osfs and memfs backends refuse.
• Without the rename, each bucket gets a fresh generation on its first multipart request after the upgrade. An upload started before then answers NoSuchUpload on UploadPart, CompleteMultipartUpload and AbortMultipartUpload, and ListMultipartUploads does not report it — no request can finish it or clean it up. Its parts are freed by the expiry sweep, which ignores the generation: within S2_SERVER_MULTIPART_MAX_AGE (24 hours by default), or never if that is negative, in which case remove the upload's directory under <root>/.multipart/ by hand. The old .meta/<bucket> markers are then inert — no listing reports them, they are not buckets, and on osfs and memfs they cannot be read back. On an osfs root, <root>/.meta/ holds nothing but those markers — a bucket's own sidecars live at <root>/<bucket>/.meta/ — so rm -rf "$S2_SERVER_ROOT/.meta" removes them and nothing else.
• memfs has nothing to migrate, and Buckets is documented as fs-family only, so there is no s3, gcs or azblob case.
• Nothing else about the layout changes. .buckets is not a valid bucket name, as .meta and .multipart were not: every name beginning with a dot is refused.

Library users

• A Storage that wraps an osfs or memfs root works as an s2-server root again. v0.18.1 refused Buckets.Create on one, because the server reached the backend's sidecar directory through a concrete-type assertion; it no longer reaches it at all.
• fs.SubSidecar is deprecated and removed in v1.0.0. Only an osfs or memfs storage itself, never a wrapper around one, ever satisfied it.
• server.Buckets now holds a sync.Mutex, so copying one by value — bs := *srv.Buckets — fails go vet's copylocks check. Hold the pointer that Server.Buckets already gives you.

Storage implementers

• Nothing changes. The name contract is the one v0.18.1 introduced.

Full Changelog

v0.18.1...v0.18.2