ObjectFS v0.13.0

Two things that existed and did nothing now do something.

cluster.enabled: true started a gossip ring and stopped there: the adapter never constructed a
ClusterManager, FileSystem had no coordinator, and AnnounceKey/QueryKeyOwnership returned
ErrNotSupported from stubs. A cluster's nodes could see each other and could not use each other. Now a
write invalidates what peers have cached at the ETag it wrote, a cache miss learns from peers which keys
are hot and reads further because of it, and a joining node is told what to warm before it serves its
first request. The bytes still come from S3 — peer data fetch is not in this release, and [#399] is why:
a sealed gossip datagram tops out at 5802 bytes of payload, so the datagram carries the metadata and
S3 carries the object. That is the honest version of the feature rather than a 128 KiB read that fails as
a 30-second timeout on a different host.

The observability half is the same shape. internal/awsrates held request prices verified against the
live AWS Pricing API and every path from a rate to a user was severed — a cost package with no
importer, a RecordCost with no caller, a report behind a config key no mount could set. Latency
percentiles were declared and never assigned, computed from a histogram that was ms % 100 rather than
a bucketing, so 50 ms and 250 ms shared a bucket. A Transfer Acceleration fallback was permanent for the
life of the mount, invisibly. Compression ran zstd over BAM, tar.zst and JPEG and then discarded the
result. Each of those was a feature the documentation described, and none of them could be observed or
switched back, which is the difference this tag is about.

Milestone v0.13.0 — Cache Warming & Operational Excellence closed at zero open, 20 issues.

Cost accounting is worth one caution: the request counts come from a middleware in the AWS SDK's
Deserialize step, not from the wrapper layer, because those two disagree — a 5 GB PutObject is one
wrapper call and 641 requests to S3. The figures are what S3 was asked for by this mount, per attempt,
excluding 5xx. They are not a substitute for Cost Explorer, and the metric's help text says so.

Added

• Data that arrived compressed is no longer compressed again ([#184]). With compression enabled, a
  write above min_size whose leading bytes name a compressed format — gzip, zstd, bzip2, lz4, xz, zip,
  PNG, JPEG, GIF, WebP, MP4, Matroska, Ogg, MP3, PDF — is stored as-is and the codec is never entered.
  These are the formats a research bucket is mostly made of, and a BAM counts: BGZF is gzip per block, so
  its first bytes are gzip's.

  Measured on an Apple M4 Max at zstd level 3, six runs per case compared with benchstat: an 8 MiB zstd
  frame goes from 1.85 ms and 33 MiB of allocation to 3 ns and none, 1 MiB from 237 µs to 3 ns,
  64 KiB from 7.4 µs to 3 ns. Data the skip does not apply to is unchanged — 1 MiB of compressible text is
  91.5 µs against 90.4 µs, which is the row that says the check pays for itself, since a gate that taxes
  the compressible path is not a win. Reproduce with BenchmarkCompressAlreadyCompressed,
  BenchmarkCompressCompressibleText and BenchmarkGateVersusAnalyze in internal/compression.

  The gate is a magic-byte comparison at 17 ns, not the full analyzer. Analyze also runs a Shannon
  entropy pass over its 4 KiB sample, which costs 3.8 µs and was +53% on a 64 KiB text write for a
  figure the decision does not use — so the write path calls a new AlreadyCompressed, and both it and
  Analyze classify through the same classifyByMagic, leaving one authority on what "already
  compressed" means. The check sits after the min_size floor, so an object below the floor is not
  sampled, and after the enabled check, so a mount with compression off samples nothing at all.

  Not done, deliberately: nothing is skipped on entropy. High entropy is evidence that compression
  will not help, not proof, and no threshold for it has been measured here — declining to compress on a
  guess is worse than spending the pass. Formats whose compression is internal to the container (CRAM,
  Parquet, ORC, HDF5, Zarr) are also not skipped, since the container's magic bytes say nothing about its
  contents; docs/features/compression.md now sorts the formats into the two lists explicitly, and
  TestAlreadyCompressedMatchesTheDocumentedFormatLists fails if that page and the table disagree.

• A mount now publishes what it is spending at AWS ([#226]). objectfs_s3_cost is a new gauge family
  carrying billable request counts by pricing group, bytes retrieved, three dollar figures, and — beside
  each — the rate it was computed at, labelled with the region the rates were resolved for and the storage
  tier they belong to. Before this, internal/awsrates held a rate table verified against the live AWS
  Pricing API by an integration test, and every path from a rate to a user was severed: the
  access-pattern report was gated behind a config key no mount could set, internal/cost had zero
  importers, and metrics.RecordCost had no caller anywhere.

  The requests are counted in the AWS SDK's own response path, not at the wrapper layer, and that is
  the substance of it rather than a detail. The eight places this backend records metrics are not
  one-to-one with billable calls: a 5 GB PutObject is one wrapper call and 641 requests to S3 — one
  CreateMultipartUpload, 639 UploadParts, one CompleteMultipartUpload — and a large CopyObject,
  a parallel ranged read and the capability probes fan out the same way, one of them counted nowhere at
  all. Every one of those errors is in the direction that flatters. A middleware in the SDK's Deserialize
  step, installed on every client this package builds including the connection pool's, cannot disagree
  with what S3 received because it is what S3 received. It counts per attempt, since the retryer
  re-enters that step; it requires a response, since a connect or DNS failure never reached S3 and is not
  billed; and it counts only statuses below 500, because AWS does not bill for its own server errors
  and does bill 4xx — a HeadObject returning 404 is a charged read, and 5xx during an S3 event would
  otherwise inflate the figure at the moment an operator is looking at it hardest.

  Four pricing groups, not AWS's two request tiers: writes, lists, reads, and the free operations. Lists
  are separate from writes even though the published price list calls both "Tier1", because on
  DEEP_ARCHIVE a PUT is $0.05 per 1,000 and a LIST is $0.005 per 1,000 — a factor of ten, and pricing
  a directory traversal at the write rate would overstate it worst on the class where the error is largest.
  Classification is by operation-name prefix and an unrecognized operation counts as a write, so an
  operation this code has never heard of makes the reported cost too high rather than too low.

  What the figures are is stated narrowly, because a cost number that implies more than it knows is worse
  than none. They are this process's spend since it started, at list prices for the first volume band:
  not a bill, and not a reconciliation of one. bytes_stored is what this mount has uploaded, not the
  bucket's size — nothing lists the bucket, since that would be a billed request per scrape to publish a
  metric. Every figure is monotonic and there is no reset, because rate-of-change is the form a useful
  alert on cost takes and a counter something can zero is one that will be zeroed. A fresh mount reports
  read_requests of 1 before any filesystem work: NewBackend's health check is a HeadBucket, AWS bills
  it, and the tests assert that 1 rather than asserting zero.

  Verified against a recording proxy rather than against the counter's own arithmetic: the tests compare
  what the tally counted with what the S3 endpoint actually received, including a multipart write where
  both sides must agree and the count must exceed the number of parts, and an injected 500 where the
  counted total must equal the observed total minus the server errors. The dollar assertions state AWS's
  published rates as literals and divide in the test, per the lesson from [#220]: the old tests passed
  1024*1024*1024 bytes, called it one GB, and asserted the per-GB rate came back — an expectation that
  holds under both the right divisor and the wrong one. Byte-to-GB conversion goes through
  awsrates.GBFromBytes throughout, and the mutation to a binary divisor is caught with a ratio of
  0.9313, which is exactly the 7.4% gap.

• A cache miss now reads further when a peer holds more of the object ([#142]). A read that misses
  asks the cluster who holds the key and, if some peer holds a range reaching past what was just read,
  reads ahead to the end of that range — capped at 4 MiB per miss. The bytes come from S3, not from the
  peer: a gossip datagram carries 5802 bytes of object at the default limit, so a 128 KiB read is 21×
  over and fails as a 30-second timeout on the requesting host ([#399]). What a peer's claim is evidence
  of is that the key is hot in this cluster, which is the one thing a cold node cannot learn on its own,
  and it is worth acting on even when the bytes still cost an S3 read.

  A claim is acted on only at the version this node is reading. A holder's range describes the object
  it cached; against a different version the object may have a different length entirely, so warming on it
  would fetch bytes chosen by a claim about something else. A version mismatch — or no known version on
  either side — warms nothing, which costs exactly the read the application asked for. Every claim at the
  matching version is considered and the furthest-reaching one wins, since holders arrive freshest-first
  and freshest is not furthest.

  Warming goes through the existing read-ahead queue, so it inherits clamping to ...

ObjectFS v0.12.0

Coordination stops pretending. ObjectFS's distributed layer had a consistency taxonomy, a Raft log,
and a CacheReplicator; what it did was send the same PUT to N nodes writing the same key and call
majority success linearizable. That is replaced by compare-and-swap on the object store itself —
Backend.PutObjectIf asserts a precondition, internal/coord's lease re-asserts the CAS on every
guarded action, and an endpoint that cannot honour a precondition is refused rather than downgraded.
The code that simulated the guarantee is deleted rather than left as a fallback, because a fallback
here reports success to every contender for a lease, which is the outcome the mechanism exists to
prevent.

This tag carries two closed milestones: Distributed Foundations & Install Simplicity (24 issues)
and Test Harness, Coverage & Build Hygiene (24 issues). Both were at zero open before it was
cut.

The second is what makes the first credible. Four SDKs shipped in this repository and not one of
them compiled: the JavaScript SDK had 48 tsc errors, the Java SDK four mvn compile errors, and
the C SDK a maximum-length S3 key that came back one byte short — each surviving because no CI job
ran the compiler. Four build tags carried code nothing built. Ten tests in tests/fuse_test.go
asserted against the mock they constructed rather than against the filesystem they discarded. A
release that adds a coordination primitive on top of that has no basis for the claim, so the gates
came first: every build tag compiles in CI, every SDK builds and tests, the lint backlog is 570 →
299, and the four suites that could not fail are gone rather than repaired.

Two findings are worth reading before deploying. Ceph RGW ≤ 19.2.0 implements conditional writes
partially — it answers 412 for a key that does not exist, rejects the quoted ETag it just
returned, and ignores preconditions on CompleteMultipartUpload, so a conditional write large
enough to be multipart is silently unconditional. The mount-time capability probe now detects this
and refuses; docs/design/conditional-write-compatibility.md records the full matrix, measured
against real AWS, MinIO and RGW endpoints rather than read from documentation. Separately, gossip
had no message authentication, and a cluster will not start without a shared secret.

What changed

• Added — 19 entries
• Fixed — 75 entries
• Changed — 13 entries
• Removed — 8 entries
• Security — 2 entries

The full text of each entry is in CHANGELOG.md, under [0.12.0].
It is long by intent: an entry records what was wrong and how the fix was verified,
which is the evidence a filesystem release has to carry.

---

Verify a download:

sha256sum -c objectfs-<platform>.tar.gz.sha256

Container image: ghcr.io/scttfrdmn/objectfs:0.12.0

Full changelog: https://github.com/scttfrdmn/objectfs/blob/v0.12.0/CHANGELOG.md

ObjectFS v0.11.0

POSIX completeness and write-path safety: the operations a user reaches for first — rm, rmdir,
mv, chmod — do the thing rather than returning an error or, worse, succeeding without acting. All
28 issues on the milestone closed before this tag was cut.

Two themes run through it. The first is that a configuration key should select something: nine
blocks that a loader had never read now reach the code they name, and the pricing region — read at
exactly one line, and only to label a summary — now picks the prices, so a mount in sa-east-1 no
longer reports us-east-1's rates under its own region's name. The second is that a fix is paired
with the mechanism that fails if it recurs, because most of what this release corrects was invisible
to the compiler, to vet, and to lint: a config key nothing reads, a cost figure nothing checks, a
gate that cannot fail.

This is the first release cut from the merge commit that closes its milestone, which is the rule
v0.10.3 established after two tags shipped without the work they were named for.

Added

• rm and rmdir work. Unlink and Rmdir delete the object rather than returning EROFS
  ([#163]). The stub they replace was itself a fix — go-fuse defaults an unimplemented
  NodeUnlinker to success, so before it rm exited 0, the kernel dropped the inode, and the
  object stayed in the bucket billing with no path that reached it. Three details are load-bearing
  and are pinned by tests:

  • Deleting a file discards whatever the write path still holds for it. echo x > f; rm f is
    ordinary and the kernel does not guarantee a flush before the unlink, so a surviving dirty range
    would be PUT back by the next flush or by the unmount — the file returning from the dead at its
    written size, with no error anywhere.
  • rmdir refuses a non-empty directory with ENOTEMPTY. S3 has no directories, so removing a
    prefix's marker object while objects remain under it would succeed at the storage layer and leave
    every one of them present, billing, and unreachable through the filesystem.
  • A missing file is ENOENT, not success. The backend's DeleteObject no-ops a key that is not
    there — S3's contract — so absence is checked explicitly, and it is checked against both the
    bucket and the write path: Create records attributes without a PUT, so a just-created file is
    real and visible to stat with no object behind it yet.

• mv works, and the README says exactly how far short of POSIX it falls. Rename copies
  server-side and then deletes, per object; renaming a directory moves everything under its prefix
  ([#164]). Before this, go-fuse's default for an absent NodeRenamer answered every mv with
  ENOTSUP. Six properties are load-bearing, each pinned by a test that was verified by mutation —
  the implementation was broken in that specific way and the test watched to fail:

  • Each source object is deleted only after its own copy has succeeded. An interruption
    therefore leaves the data at the old name, the new name, or both — never at neither. Duplicated
    data is an operator's cleanup problem; missing data is not recoverable, so the ordering is fixed
    in that direction deliberately, and a partial directory move is resumable by re-running the same
    mv.
  • The copy is server-side. Reading and rewriting through the process would make renaming a
    10 GiB file cost 20 GiB of transfer, and renaming a directory that times the object count.
    Objects above S3's 5 GiB single-part CopyObject limit route through UploadPartCopy.
  • The write path is flushed before the copy runs. A copy acts on objects, so a file whose only
    content is dirty ranges in memory is invisible to it: echo hi > a; mv a b would have copied a
    key that did not exist yet, deleted the source, and then flushed the pending ranges back to the
    old name — the file landing at neither name the user asked for.
  • The moved node is repointed, recursively for a directory. go-fuse's MvChild re-parents the
    same inode, so the FileNode survives a rename holding the path it was constructed with. A
    stored path is stale the moment a rename succeeds, and the consequence is silent in a way nothing
    would catch: after mv a b, a write to b flushed to a and recreated the source the rename
    had just deleted. A stale key is a valid key, so every S3 call succeeds.
  • Prefix matching is on a path boundary. mv dir dir-new must not move dir2/file. That is not
    a hypothetical spelling mistake — it is the same defect the cache's keyMatches had, found in the
    same audit.
  • renameat2's RENAME_EXCHANGE and RENAME_NOREPLACE are refused with EINVAL, not
    approximated. Both are atomicity promises copy-then-delete cannot keep, EINVAL is what the
    kernel and libc expect for an unsupported flag, and mv and Git fall back correctly on it. A
    foreign newParent is EXDEV, which is what go-fuse's own LoopbackNode answers.

  There is no atomic alternative to reach for. S3's RenameObject, added in 2026, is
  directory-bucket (S3 Express) only, and object annotations — which ObjectFS needs for POSIX
  attributes — are unsupported on directory buckets, so the two features are mutually exclusive. The
  README now carries a Rename is not atomic section stating what a concurrent reader can observe,
  what an interruption leaves behind, and which tools that breaks: anything relying on the
  write-temp-then-rename idiom for atomic replacement is not safe here between concurrent writers.

• types.Backend.CopyObject, a server-side copy that preserves content encoding, content type,
  storage class, and user metadata. Each of the four is a requirement rather than a nicety, and the
  interface documentation says which failure each one prevents: the read path dispatches decoding on
  the stored Content-Encoding and fails closed on one it cannot handle, so dropping it would leave a
  compressed object permanently unreadable with its bytes intact; the storage-class default is
  STANDARD, so dropping it would silently promote the object out of the tier being paid for; and
  POSIX mode, ownership, and mtime live in user metadata and nowhere else, so dropping it would reset
  a file's permissions, which is not a thing rename does. Encryption is applied from configuration
  rather than copied from the source, so a key rotation reaches renamed objects.

• internal/testaws: a DirectoryMarkerDelete capability probe, and RequireDirectoryMarkerDelete
  to skip on its absence. Deleting a dir/ marker while dir/child still exists panics the substrate
  emulator, because its object store is a filesystem abstraction where dir/ is the directory
  holding the child, and removing it orphans the child (filed as
  scttfrdmn/substrate#534, reproduced against
  afero alone with no substrate involved). In S3 a key is an opaque string and dir/ is an ordinary
  object, so this is the emulator's property and not ObjectFS's — which is the reason for a runtime
  probe rather than an assertion either way. The alternative was to assert the emulator's behavior as
  expected, which would encode a dependency's bug as this project's contract.

• internal/storage/s3/copy_live_test.go (-tags=integration) covers the multipart copy path against
  real AWS, at real part sizes. It has no hermetic equivalent: substrate dispatches on
  x-amz-copy-source before checking uploadId, so it answers an UploadPartCopy as a whole-object
  copy with a 200 (substrate#532). That leaves the branch where a mistake is most expensive resting on
  nothing but reading — abandoned multipart parts are billed and invisible to ListObjects. Two
  shapes, both deliberate: a legal three-part copy with a short final part, which is where
  inclusive/exclusive range mistakes surface, and an illegal one that S3 rejects at
  CompleteMultipartUpload, which is how the abort path gets a failure arriving after every part has
  already uploaded.

• A fuse: section in the configuration file, and it is the first one any loader has read
  ([#180]). direct_io, keep_cache, and sync_read. Nine fields on internal/fuse.MountOptions and
  internal/fuse.Config carried yaml tags for a whole release and were decoded by nothing, because
  config.Configuration had no fuse key at all — so a fuse: block in a config file was silently
  discarded, and the two flags that reach the kernel per-open were settable in Go and returned as the
  literal 0 from every Open. All three new fields default to false, false is the kernel's own
  behavior for each, and NewDefault therefore names no fuse section: the zero value is the
  default, and a second place for it to live is how the last set drifted.

  Each of the four seams between the YAML key and the value the kernel receives is now asserted by a
  test verified by mutation — the mapping was deleted or reverted to v0.10.0's code and the intended
  test watched to fail. That is the point of the change rather than a detail of it: every one of those
  nine fields was correct at the layer that declared it, and died at a boundary no test crossed. The
  adapter mapping in particular passed its package's whole suite with the three assignment lines
  removed, which is why it was extracted into a method that can be called without a mount.

  What the flags do to the kernel — whether a second read(2) at the same offset reaches the
  filesystem, whether cached pages survive open(2) — cannot be observed without /dev/fuse, so it
  lives behind a fuse_mount build tag with a make test-fuse-mount target. CI compiles the tag it
  cannot run, because a build tag nothing compiles is how four others in this repo came to carry code
  that does not build ([#240]). Those tests fail rather than skip when the device is ab...

ObjectFS v0.10.3

Part 4 of the v0.10.0 audit: say only what the code does, and bill accurately. The audit found that
documentation, cost figures, and repository metadata each asserted things no mechanism checked, so
most of what follows is a correction paired with the gate that fails if it recurs — five of them,
now running on every PR.

This release exists because of a numbering defect worth stating plainly, since it is the second
instance of the same one. v0.10.2 was tagged when the first of this milestone's twelve issues
closed, and the remaining eleven landed after it — so the tag published under that number holds the
packaging fix and none of the work below, exactly as v0.10.1 was tagged two hours before the
module-path fix it was cut for. A tag is not a promise that can be revised: it is already in the
GitHub releases list and cached by the Go module proxy, which resolves a version to one tree
forever. So the fix is a new number rather than a moved tag, and the rule that prevents a third
instance is to cut the tag from the merge commit that closes the milestone, not from the one that
opens it.

Added

• A gate that fails when CHANGELOG.md's version headings and its link definitions disagree. Keep a Changelog puts each release in a bracketed heading and defines the link separately at the bottom of the file — two hand edits per release with nothing connecting them, and markdown fails silently in both directions. An undefined reference renders as the literal text [0.10.2] instead of a link; a definition with no section renders as nothing. Neither breaks a build, fails a lint, or produces a visibly wrong page, so the only witness is a reader noticing a heading stopped being clickable, which is not a thing readers report. Both halves had already broken: [0.10.2] was never defined at all, and [Unreleased] still compared from v0.10.1 — so the link that answers "what is on main but not released" spanned two releases and 52 entries of already-released work. internal/config/changelog_test.go checks three properties: every section has a definition and every definition has a section, [Unreleased] compares from the version constant, and each release's diff starts at the release immediately before it. The third exists because that failure is the one that renders and resolves — copying the previous definition and editing only the right-hand side produces a real GitHub diff covering more releases than the section it is attached to. All four failure modes were verified by mutation, which is also how the orphan-definition message got fixed: it printed the URL where the version belonged. This is the same defect the release itself is about, one file over — a fact restated in a second place with no mechanism to notice the two have drifted
• A gate that fails when documentation names a Go symbol or a CLI flag that does not exist. internal/config/docs_symbols_test.go extracts pkg.Symbol references from fenced Go blocks — checked against the packages that same block imports, parsed with go/ast — and objectfs command lines from shell blocks, checked against the flags cmd/objectfs/main.go actually declares. This is the mechanism #182 asked for, and the point is that it fires at authoring time: correcting the nineteen files that issue cataloged only resets the clock, since a fenced code block is a string as far as the compiler, vet, and lint are concerned. It found eleven defects on its first run, listed under Fixed below, and a companion test compares the flag list against main.go so the two cannot drift apart silently. The admission rule — which references get checked — was chosen by measuring three candidates rather than guessed: checking every lowercase.Uppercase in every Go block gives 93 findings of which 3 are real (s3.Client is the AWS SDK, errors.Is is the standard library), file-scoped imports give 5 of which 3 are real, and block-scoped gives 3 of 3 with no false positives. Its one known blind spot is stated in the test rather than left to be discovered: a continuation block that uses a package imported by the block above it is not checked
• A gate that fails when documentation links at a page that does not exist. internal/config/docs_links_test.go extracts every relative markdown link from every tracked markdown file and resolves it on disk — relative paths against the linking file's directory, root-absolute paths in docs-platform/ against VitePress's routing rule, where /guide/installation is served from guide/installation.md and /api/ from api/index.md. This is #208's mechanism, and it is a Go test rather than the link checker in CI that issue proposed for three reasons recorded in the file: it needs no network and no new tool, so pre-commit and CI check at identical fidelity; it sits with the gates a contributor already satisfies; and an exemption can carry its reason in code, the way docsExemptFromConfigSchema does. It found 45 dead links, not the 24 the issue catalogued — because #208 was written by walking docs/, and two whole classes live outside it: 13 links into SDK examples/ directories that have never existed, and 8 root-absolute VitePress routes. That gap is the finding, and it is the same shape as docs_test.go's nestedSectionNames: scoping a gate to where the defects were already known is how the next cluster stays invisible. A link target is a path, not a symbol, which is why the symbol gate above cannot see it — a link written as [tuning] followed by (./perf.md) is prose to the compiler, to vet, and to lint, and stays prose after the file is renamed. A third test asserts the walk's reach rather than its findings, and it earns its place: a mutation that made the link regexp match nothing left the resolving test passing on zero links and green, and only the reach test caught it
• A gate that checks mkdocs.yml's nav against the tree, in both directions. TestMkDocsNavMatchesTheTree asserts that every nav entry has a file and that every page under docs/ is either in the nav or exempt with a stated reason. Both directions, because that is how the defect ran: 47 of 50 entries pointed at no file, and 14 of the 17 pages in the tree were missing from the nav. Checking only that entries resolve would have left the orphans, which is the half a reader loses — a page absent from the nav is a page nobody finds. A nav entry is a link target with a different syntax, and that syntax is why it went unchecked: the link gate walks markdown, and nav: is YAML. It is a line scan rather than a YAML parse for a stated reason — mkdocs.yml carries !!python/name: tags for the emoji and superfences extensions, so decoding it needs a custom resolver or unsafe mode, and the nav is a flat list of - Title: path.md lines that needs neither
• docs/features/compression.md — what transparent compression costs, and when it saves nothing. The question it answers is the one #186 was filed for: project-level compression saves bandwidth and end-to-end latency, so what else does it buy? Less than you would expect. It names four costs, each measured rather than asserted: a compressed object is not readable by anything but ObjectFS (aws s3 cp and boto3 both write the raw zstd frame to disk with a successful exit status — no error to notice); a 4 KiB read of a compressed object transfers the whole stored object, which is 1,836× / 7,344× / 29,380× amplification at 16/64/256 MiB; enabling compression turns off parallel range reads for every object in the bucket, compressed or not; and on the three tiers with a 128 KB billable floor, compressing a 100 KB object to 40 KB changes the invoice by zero. Byte counts are presented as the result and wall-clock only as an aside, for a reason stated on the page — bytes are a property of the design, latency is a property of the day, and the audit's 15.6×/43×/216.5× and this page's 3.0×/5.0×/12.3× are the same defect measured on different days. Every figure is either linked to the AWS page that publishes it or carries its bucket, region, date, and payload, which is docs-platform/index.md's standard after its hardcoded chart was removed. Two of the numbers #186 itself specified are wrong, and the page states what AWS publishes instead: AWS applies no minimum billable object size to GLACIER, DEEP_ARCHIVE, or INTELLIGENT_TIERING. The archive classes' 40 KB is metadata added per object (32 KB at the archive rate, 8 KB at Standard), which points the opposite way from a floor — compression does reduce the bill there, it just cannot touch the surcharge, which is about 23× the payload for a 10 KB object on DEEP_ARCHIVE. Writing the page found three defects, filed as #228, #229, and #230
• A gate that fails when .github/labels.yml and the repository's labels disagree — in both directions. internal/config/labels_test.go is the fifth mechanical gate, and it exists because the file is a hand-maintained description of state held on GitHub and nothing compared the two, so they had drifted by nine labels. Both directions, because only one is intuitive: all nine existed on GitHub and were absent from the file, none the other way, so a sync that creates labels from the file is green on every one of them — it has nothing to create. That is the failure mode #190's own acceptance criteria name, and the test for the gate is the one they specify: create a label on GitHub without touching the file and confirm the job notices. Verified by doing exactly that, with a throwaway zz-drift-probe. Colors and descriptions are compared too, not just names — a label the file describes differently from the label that exists is drift with a longer fuse, because the name still filters correctly and nothing looks wrong. #190 proposed a paths:-filtered sync job that runs when labels.yml changes; measurement is why this one runs unconditionally instead. Every drift this repository has had originated on GitHub — tw...

ObjectFS v0.10.2

A packaging release, cut for one reason: v0.10.1 was tagged two hours before the module-path fix
merged, so the published tag still declared module github.com/objectfs/objectfs and
go get github.com/scttfrdmn/objectfs@v0.10.1 failed with module declares its path as — the exact
defect #213 was filed for. The fix existed on main and in no tag, which from a user's position is
indistinguishable from not being fixed. Everything else here is the packaging and contributor-path
work that landed alongside it.

Added

• SECURITY.md — a security policy, with private vulnerability reporting enabled on the repository so a finding has somewhere to go that is not the public issue tracker. It documents what a reader cannot get from the code quickly: that the trust boundary is the mounting host and ObjectFS enforces no authorization of its own, that two unauthenticated HTTP listeners bind all interfaces by default (:8080 metrics and debug endpoints, :8081 health) with the switch that turns each off, that mode: off is the encryption default and what changed after the withdrawn v0.10.0 at_rest key, and both stated limits of the SHA-256 read verification — a partial read is not verified, and an object with no recorded checksum verifies trivially. Every claim in it was verified by execution rather than read off the configuration schema, which is how the two listener defects below were found

Fixed

• The test harness could record a request after the client already had the response, so its own assertions were load-dependent. internal/testaws proxies every request and logs it, and the read-path suite asserts on that log: bytes transferred and GETs issued are how read amplification and cache behaviour are measured, because neither the AWS SDK nor the emulator reports them. The log entry was appended after proxy.ServeHTTP returned — but the proxy writes the body to the socket inside that call, so a client could hold every byte of a response whose request was not yet recorded. Measured at 45–70 of 640 concurrent ranged reads. The visible symptom was in a different package: internal/fuse TestShortFileIsServedFromCache failed on its precondition — "the first read issued no GET" — which reads as the read path serving bytes from a cache the fixture had just created empty, in a test whose entire subject is cache correctness. One CI run in seven. Requests are now published on arrival and their response fields filled in on completion, with the accessors waiting for anything still in flight; verified in both directions, since a regression test that cannot fail proves nothing
• The module could not be imported under the name it gave for itself. go.mod declared module github.com/objectfs/objectfs, and the code lives at github.com/scttfrdmn/objectfs. Go resolves an import path by fetching that path, so go get github.com/scttfrdmn/objectfs failed on the mismatch between the path requested and the path declared, while the declared path is a different project — an unrelated Python repository from 2017, 28 stars, last pushed 2019, in a single-repo organisation created the same day. Nothing published has ever existed at the declared path, which is why pkg.go.dev had nothing to index and the Go Reference badge rendered empty. The path is corrected in go.mod and in all 154 files that named it — 132 Go files, plus the goimports local-prefixes setting in .golangci.yml, the Dockerfile image-source label, and the repository URLs in the Python and JavaScript SDK manifests, which pointed contributors at the wrong project. Verified by building an external consumer module against the corrected path, rather than by grepping for the string. This is breaking for any code that imported the old path, though nothing could have: it was never fetchable
• Dependabot could not update Go dependencies, and had never merged anything. Two unrelated defects presenting as one symptom. The Go ecosystem failed on twelve consecutive weekly runs while docker succeeded in the same runs — Dependabot aborts per-ecosystem, so one broken ecosystem is silent unless the run list is read. The cause was upstream and is now resolved: proxy.golang.org had no .mod for the pinned cargoship version, Go fell through to direct git, and git reported the proxy's 404 as could not read Username for 'https://github.com' — an authentication message for what was not an authentication failure, which is what sent the previous diagnosis after a credential that was never missing. Separately and more consequentially, .github/dependabot.yml labelled every PR automerge and that label did not exist; Dependabot drops unknown labels without reporting it, and every approve and merge step in dependabot-automerge.yml was gated on it, so 46 PRs were opened and none were ever merged. The label is now declared in .github/labels.yml alongside the four others the config names
• .github/dependabot.yml: maven and npm ecosystems for sdks/java, docs-platform, and sdks/javascript. Eight open Dependabot alerts — five against jackson-databind, three against vite, three of the eight high severity — were against manifests no ecosystem entry covered, so nothing could act on them. sdks/javascript is included because CI runs npm install && npm test there on every PR, which makes its dependencies executed code. A ceiling worth stating: the npm security updates still cannot apply, because neither directory commits a lockfile and Dependabot cannot determine the installed version without one (#214)
• .github/workflows/dependabot-automerge.yml waited on check-regexp: (test|lint|security).*, which is case-sensitive and start-anchored, so it matched 2 of the 9 checks CI produces and ignored coverage, config-examples, every cross-build matrix leg, sdk-metrics, fuzz-smoke, and Security Scan. The wait step is removed rather than corrected: which checks must pass now lives in branch protection on main, which also governs human PRs and cannot drift from a regexp in a workflow file. Native auto-merge is enabled on the repository, without which the --auto flag would have failed even once the label matched
• docs-platform/docker-compose.yml was not valid YAML. Two healthcheck entries put a bare URL inside a flow sequence, where the scanner reads http as a plain scalar and then meets : in place of , or ]. Docker's own parser is lenient enough to accept it, so it went unnoticed — but pre-commit run check-yaml --all-files failed on the file, which is the first thing a new contributor runs
• .gitignore did not cover coverage/, the directory make coverage writes into. The three bare coverage.* filenames only match a profile written to the repository root, which no target produces
• scripts/setup-hooks.sh — the first command CONTRIBUTING.md tells a contributor to run — failed on any current macOS or Debian host, and exited 0 having installed nothing. Five defects: pip3 install pre-commit ran first and dies with externally-managed-environment on a Homebrew or Debian Python (PEP 668), and because the installer was an if/elif chain testing only whether each command exists, a failing pip3 never fell through to the brew branch that would have worked; the failure happened inside a condition, so set -euo pipefail did not catch it; it installed gosec from github.com/securecodewarrior/gosec, which is a 404 (the real module is securego/gosec, which security.yml already uses); it pinned golangci-lint v1.55.2 against a version: "2" config only v2.x can parse, handing contributors a lint failure that looks like their fault; and it wrote a .golangci.yml if none was present, containing linters removed from golangci-lint years ago — now that a real config is committed, that branch would have overwritten it with an unusable one. Each install method is now tried until one succeeds, pipx first, every path verifies the command is on PATH afterwards, and the golangci-lint check is version-aware rather than presence-only. It also no longer overwrites .git/hooks/pre-commit with a hand-rolled wrapper that blocked any commit touching a line matching fmt.Print or TODO, including inside a string literal or a comment explaining why a TODO is deliberate
• make printed four overriding commands for target warnings on every invocation, including make help. BUILD_DIR := build and COVERAGE_DIR := coverage made the directory-creation rule read bin build dist coverage:, colliding with the real build and coverage targets. The build worked — the later recipe wins, and both are .PHONY — but a build system that opens with four warnings reads as unmaintained, and the names would have genuinely collided the moment one stopped being .PHONY. Replaced with a %/.mkdir sentinel rule, which keeps the pattern out of the target namespace, declared as an order-only prerequisite so writing one binary does not rebuild its siblings
• pre-commit run --all-files could not complete: the pretty-format-yaml hook crashed on import, because the pinned rev imports pkg_resources, which modern setuptools no longer ships (Python 3.14 here). Bumped to a rev that does not. Fixing it exposed a second problem worth recording, since the obvious repair is the wrong one: check-yaml is PyYAML and follows YAML 1.1, where a bare URL in a flow sequence is a syntax error, while pretty-format-yaml is ruamel and follows YAML 1.2, where it is legal — so quoting the URL to satisfy the first makes the second strip the quotes straight back off, and the two hooks disagree forever. The docker-compose.yml healthchecks are now block sequences, the one form both parsers accept and where there are no quotes left to remove
• .golangci.yml is excluded from the pretty-format-yaml hook, which damages it: the formatter dedents every block sequence to its parent's column, destroying the nesting tha...

ObjectFS v0.10.1

Every entry below is user-facing, and the release is almost entirely one thing: the defects a deep
audit of v0.10.0 found, and the harness that would have caught them. v0.10.0 is withdrawn.

Four defects in v0.10.0 were verified by execution to lose or corrupt data, and one prevented the
shipped default configuration from mounting at all. They were not independent — they clustered in
three subsystems whose designs could not express what they were asked to do, which is why this
release adds internal/vfs rather than patching six call sites. The write path could not represent
an offset write, the read cache could not hit as keyed, and the FUSE node layer was missing most of
its contract.

The reason 32,680 lines of tests across 90 files caught none of it: every one was a seam defect —
a value correctly produced at one layer and silently dropped at the boundary to the next. A mock on
the far side of a seam agrees with its caller by construction. internal/testaws and
internal/difftest exist to remove that blind spot.

What changed

• Added — 57 entries
• Removed — 16 entries
• Changed — 34 entries
• Fixed — 101 entries
• Deprecated — 1 entry

The full text of each entry is in CHANGELOG.md, under [0.10.1].
It is long by intent: an entry records what was wrong and how the fix was verified,
which is the evidence a filesystem release has to carry.

---

Verify a download:

sha256sum -c objectfs-<platform>.tar.gz.sha256

Container image: ghcr.io/scttfrdmn/objectfs:0.10.1

Full changelog: https://github.com/scttfrdmn/objectfs/blob/v0.10.1/CHANGELOG.md

v0.10.0 — WITHDRAWN, do not use

⚠️ This release is WITHDRAWN — do not use it

A deep audit of v0.10.0 found defects that prevent the shipped default configuration from mounting
and that silently lose or corrupt user data. v0.10.1 is in progress. Until it ships, no tagged
version of ObjectFS should be used for data you care about.

The three that matter most

C1 — the default configuration cannot mount.
internal/config/config.go defaults compression.algorithm to gzip, but
internal/compression/codec.go implements only none, zstd, and lz4. Every layer that reads
config treats gzip as valid — only the codec factory disagrees — so objectfs s3://bucket /mnt
exits with Failed to start adapter. examples/config.yaml ships the same broken value.

H7 — offset writes truncate the object.
The write-buffer flush callback in internal/adapter/adapter.go is handed (key, data, offset) and
calls backend.PutObject(ctx, key, data), discarding the offset. PutObject is a whole-object
replace, so:

$ dd if=/dev/zero of=f bs=1M count=1        # 1 MiB file
$ printf X | dd of=f bs=1 seek=1048575 conv=notrunc
$ ls -l f
-rw-r--r--  1 byte                          # the other 1,048,575 are gone

Non-contiguous writes — SQLite, mmap writeback, tar, HDF5 — return EIO instead. Flush errors are
recorded to a stats counter and never returned, so close(2) reports success after a failed upload.

C4 — read amplification on every object when compression is enabled.
internal/storage/s3/backend.go decides whole-object-versus-ranged fetch from the compression
configuration rather than from the object being read. A ranged read of any object — including
objects never compressed, objects below min_size, and objects written by other tools — downloads
the whole object, and parallel reads are disabled bucket-wide. Measured against real S3 in
us-west-2 with a fixed 4 KiB read:

object size	compression off	compression on	penalty
16 MiB	123 ms	1.92 s	15.6×
64 MiB	117 ms	5.03 s	43.0×
256 MiB	227 ms	49.2 s	216.5×

A 4 KiB read of a 10 GiB object transfers 10 GiB.

Also withdrawn for

• Silent corruption when the codec configuration changes. Decompress in
  internal/compression/s3_integration.go returns the payload unchanged when the stored
  Content-Encoding doesn't match the configured codec — so an object written with zstd and read
  after switching to lz4 emits the raw compressed frame with exit status 0. The objectfs-sha256
  metadata this very release added is written and never read, so nothing catches it.
• The read cache cannot hit and is never invalidated. The cache key includes the requested
  length, so the Lookup metadata cache never hits (one S3 HEAD per path component per stat,
  forever), short reads at EOF are uncacheable, and the 16 MB chunked cache population added in this
  release is unreachable. There are no cache.Delete calls anywhere in internal/fuse, so a read
  after a write on the same descriptor returns pre-write bytes for up to the 5-minute TTL.
• A reachable panic that unmounts the filesystem. GetObject with a negative size slices
  data[offset:offset+size] with neither bounds arm firing — slice bounds out of range [100:99].
  This kills the mount process and takes every open file descriptor with it.
• This release's headline feature is inactive in production. buildS3Config maps 6 of roughly 30
  s3.Config fields and does not map ParallelReadThreshold; NewBackend does not backfill it. The
  parallel range GET path is gated on threshold > 0, so it never runs on a real mount. PoolSize
  is likewise unmapped, leaving a zero-capacity semaphore that blocks forever in
  GetObjects/PutObjects.
• rm and rmdir reported success without deleting. Fixed after this tag (#163): the operations
  now fail loudly with EROFS rather than silently lying.
• Windows is not supported. The cgofuse build tag has never compiled. Any Windows claim in the
  v0.10.0 documentation is wrong.

Why the test suite didn't catch this

Every defect above is a seam defect: a value correctly produced at one layer and silently dropped
at the boundary to the next. The suite's 32,680 lines across 90 files mock the neighbouring layer in
each case, so these are invisible to it by construction. v0.10.1 adds a differential-testing oracle —
identical operation sequences run against ObjectFS and against the local OS filesystem, asserted
byte-for-byte — plus fuzz targets over the write path, the range/slice domain, config loading, and
compression round-trips. Those tests are being written before the fixes, so each fix lands with a
failing-then-passing test.

Track progress: issues ·
milestones

ObjectFS v0.9.0

feat: ObjectFS v0.9.0 — stub replacement & quality pass

Implements all 10 issues from the v0.9.0 audit (#118–#127):

• #118: Fix hardcoded "0.6.0" version in API server; use ServerConfig.Version
• #119: S3 applyOptimization now calls CopyObject for real tier transitions
• #120: MultiLevelCache.Warmup() fetches keys from backend instead of no-op
• #121: CgoFuseFS.GetStats() returns real atomic counters (reads/writes/etc.)
• #122: attemptAutoRecovery() now calls Recoverable.Recover() with retry logic
• #123: Add POST/GET/DELETE /api/v1/mounts REST endpoints + MountManager iface
• #124: Adapter.Stop() clears cache and stops metrics collector (was TODO)
• #125: Add benchmarks for cache, buffer, and adapter packages
• #126: Add test coverage for internal/filesystem package (mockFilesystem)
• #127: Add sync.RWMutex to Go SDK Client for concurrent safety

Closes #118, #119, #120, #121, #122, #123, #124, #125, #126, #127

v0.8.0

What's Changed

Added

• Distributed backend wiring (internal/distributed/): ClusterManager.SetBackend and Coordinator.backend wire the types.Backend S3 backend into executeLocally, replacing the in-process stub with real GetObject/PutObject/DeleteObject/ListObjects calls; nil backend returns a descriptive error instead of phantom data (#85)

• Distributed cache invalidation (internal/distributed/): New MessageTypeCacheInvalidate gossip message type, ClusterManager.SetCache / InvalidateCacheKey methods, and handleIncomingMessage dispatch that calls cache.Delete(key) on all peers within one gossip round-trip (#86)

• S3 backend benchmarks (internal/storage/s3/backend_bench_test.go): Nine benchmarks covering GetObject 1 KB / 1 MB / 10 MB, PutObject 1 KB / 1 MB, DeleteObject, ListObjects 100 / 1000 entries, concurrent Get, and latency distribution; use an in-process stub — no AWS credentials required (go test -bench=. ./internal/storage/s3/...) (#88)

• pjdfstest POSIX harness (scripts/pjdfstest.sh): Shell script that mounts ObjectFS against a test bucket, runs the pjdfstest suite for POSIX compliance validation, and unmounts on exit; make test-posix target added to Makefile (#89)

• Java 17 SDK (sdks/java/): Maven SDK with ObjectFSClient (get, put, delete, list, head, mount, unmount, isHealthy), ObjectFSConfig (builder pattern), ObjectInfo, MountOptions, ObjectFSException, NotFoundException, and a full JUnit 4 test suite using MockWebServer (#90)

Changed

• Structured logging (internal/distributed/, internal/health/, internal/fuse/, internal/adapter/, internal/cache/redis/, pkg/profiling/): All log.Printf calls migrated to structured slog.Info/slog.Warn/slog.Error with key-value attributes across 13 internal packages; log.Fatal retained in cmd/objectfs/main.go (#87)

Full Changelog

https://github.com/scttfrdmn/objectfs/blob/main/CHANGELOG.md

Release v0.7.3

Changes

• fix: resolve nine bugs found during v0.7.3 audit (42d0d37)