Skip to content

v0.5.0 — one status code, and the parsers fuzzed

Latest

Choose a tag to compare

@youndie youndie released this 01 Sep 15:32
· 11 commits to main since this release
51c116e

An S3-compatible object store on the JVM: one process, one node, one disk. Objects live as files
and the whole key index lives in memory, so what the store promises about capacity is a published
number rather than a surprise.

The niche is two-sided and both sides are the same server: a store you run on one machine behind
your own TLS terminator, and an S3 endpoint you start inside a test — where the alternative on
the JVM today is a mock, and a mock answers what it was told to answer.

What is new in v0.5.0

One behaviour changed, and the rest of the release is the reason to believe the rest of the
server.
That is worth saying plainly rather than dressing up.

A truncated percent escape in a URI — ?%, or a key ending in %zz — used to answer
500 InternalError. It answers 400 InvalidURI now.

The status is the whole point. A 5xx tells a client the server is broken and the request may
succeed on a retry; this one never will, and both aws-cli and boto3 retry a 5xx five times. So
one bad byte in a URI cost five requests and taught nobody anything. The refusal is typed at the
codec and answered in one place, because nine call sites reach it — the path, the object key, the
query, the copy source, and the components a signature is rebuilt from — and catching it beside the
parser that happened to find it would have left the other eight answering 500.

Everything else is verification. Since v0.4.0 the server gained no features, and gained this:

mutation testing a repeatable run with every survivor classified by a named reason, not a score
the code, in English 1183 lines translated, and a gate that fails if it drifts back
fuzzing every parser that reads unauthenticated bytes, ~420 million inputs
the gate 933 tests in 107 classes, against 741 at the last release

About a third of that test growth is the fuzzing corpus: an input found once is replayed by every
check from then on, under its own filename, so a finding becomes a permanent test rather than a
memory.

Two of those instruments reported that all was well while having done nothing at all, which is how
the 500 above was found in the first place — and how it was nearly missed. A fuzzer with no seed
corpus did 27 820 013 inputs against the head parser and found nothing; two seed files later it
found something on input 24, because random bytes essentially never spell <method> <target> <version> and everything behind the request line is unreachable without a seed.

Nothing else moved. Installing with your existing values gives exactly what v0.4.0 gave, and the
bochka-embedded ABI has not changed since v0.2.0.

Where it stands

518 of 744 ceph/s3-tests as it ships, 533 of 744 with BOCHKA_ANONYMOUS=1 — both
measured, at suite revision 5522d1c, and every remaining failure is classified with a reason.

That number was re-measured for this release rather than carried forward, because the one behaviour
change is a status code and the suite is made of status codes. It did not move: 226 failures, none
unclassified — 213 deliberate scope decisions, 11 that pass only behind the anonymous switch, and 2
waiting on a source rather than on work.

Beside that: four other people's clients over a real socket (aws-cli, boto3, mc, rclone),
all four body framings including the one aws-cli only sends over TLS, a crash test that kills the
JVM with SIGKILL mid-write and demands that everything the log admitted to still reads back, and a
bytecode gate holding the one property no test inside a JVM can see — that the durability barriers
are still called at all.

Install

docker run -d --name bochka \
  -u 1000:1000 -v /srv/bochka:/var/lib/bochka \
  -p 127.0.0.1:9000:9000 \
  -e BOCHKA_KEYS='youraccesskey:yoursecretkey' \
  ghcr.io/youndie/bochka:v0.5.0

Chart: oci://ghcr.io/youndie/charts/bochka:0.3.1. In a test:
io.github.youndie.bochka:bochka-embedded:0.5.0.