Repository navigation
Releases
v0.1.1 — a measured decode win, and a page that can tell a tie from a loss
v0.1.1 — a measured decode win, and a page that can tell a tie from a loss
No API change. internal/apicontract/testdata/public_api.txt is byte-identical to
v0.1.0, so this is a drop-in patch: move the pin and nothing else.
THE DECODE FIX. stallGuard.progressed proves a struct read made progress. A
positive clock advance is by itself sufficient proof — but it was evaluated
SECOND, so every ordinary Item summed all nine V2 column streams plus the rest
buffer before reaching the cheap test that would have short-circuited it. On a
10,000-struct update that is 10,000 redundant nine-stream scans.
Reordering the && operands is worth 3-4% on ApplyV1 and ApplyV2, and the
equivalence is structural rather than empirical: with &&, the boolean result is
order-independent by definition, and every operand bottoms out in a length read —
decoderRemaining -> RemainingLen -> nine .remaining() calls -> buf.Len(). Nothing
mutates decoder state, so the guard fires on exactly the same inputs. The
accompanying test was verified to FAIL on the old ordering before it was accepted.
Measured under BOTH GC regimes, deliberately: -4 to -7% decode / -3 to -4% Apply
under normal GC, -5.6% / -2 to -3% under GOGC=off. In the same investigation,
presizing the destination struct store won under GOGC=off and LOST 5-11% under
normal GC — the extra live pointer-bearing array cost more in scan time than it
saved in growth. A single-regime measurement would have shipped that as an
improvement.
WHAT THIS DOES NOT DO. It does not close the ApplyV2 gap to yjs, and nobody should
read it as fixing a slow codec. Measured in one session on one host, ApplyV2 is
about 1.12x yjs; ApplyV1 we win at 0.83x. Of our 1.80 ms decode, roughly 0.33 ms
is actual V2 column scanning and about 1.16 ms is Go object-graph materialisation.
yjs materialises the same graph eagerly, so the comparison is honest — the residual
is V8's monomorphic constructor loop against Go's large pointer-bearing structs and
interfaces. That is a property of the two runtimes, not a defect here. Integration,
where most optimisation effort went, is at parity.
THE PERFORMANCE PAGE now ships at docs/performance-status.html and has a parity
band. It previously called every ratio below 1.00x a loss — so a 0.99x dead heat
rendered red — and everything under 1.50x "thin", so a real 1.4x lead looked like a
tie. Below 0.85x is now a clear loss, 0.85-1.15x is parity, above 1.15x is a clear
win. The band is +/-15% because run-to-run spread reaches 20% on the JS side alone;
inside it the harness cannot separate the two implementations. On current data: red
26 -> 12, green 125 -> 142.
Its numbers come from a single-session four-leg run. Every ratio quoted before this
spliced a fresh go column against competitor columns measured a week earlier on a
different CPU allocation, and that splice reported ApplyV2 as 1.33x where an
in-session run puts it at 1.12x. run-all.sh's header has always warned that columns
from different windows are not comparable.
DOCUMENTATION CORRECTIONS, each of which promised more than the code delivered:
README and SECURITY.md claimed the oracle runs both directions on all thirteen
surfaces — eight do, and undo, relative positions, sync, awareness and subdocuments
are direction A only, so bytes this library EMITS are not differentially validated
on those. persistence.Deleter now states what Invalidate does not exclude before a
durable delete. Plus stale identifiers the v0.1.0 rename sweep could not reach,
because prose is not compiled.
v0.1.0 — Go-conventional naming, and a lint config that is actually enforced
BREAKING. Public identifiers now follow Go's initialism conventions:
Doc.Guid -> Doc.GUID
Doc.GetSubdocGuids -> Doc.GetSubdocGUIDs
Doc.GetXmlFragment -> Doc.GetXMLFragment
ToJson -> ToJSON on Doc, YArray, YMap, YText, YXmlFragment
Mechanical to adopt: rename the call sites. A library pitched as the standard Go
Yjs implementation should not read as a transliteration of JavaScript, and doing
this before promotion costs less than doing it after.
The rename exposed a real defect rather than only tidying names. YXmlText carried
BOTH an inherited ToJson() interface{} and its own ToJSON() string, returning
DIFFERENT strings — YText.ToString() is plain text, YXmlText.ToString() builds
the XML form. Two methods differing only in capitalisation, invisible for exactly
that reason. Each type's public method now keeps its Yjs-correct signature and
the polymorphism moved to an unexported method.
A COMMITTED .golangci.yml, so "0 issues" means the same thing on every machine
and across a linter upgrade. Fourteen linters, no exclusions section, no nolint
directives anywhere in the tree. Adopting it closed 1073 findings by changing
code rather than configuration; four linters are deliberately not enabled and are
recorded in docs/LIBRARY-GAPS.md with counts and reasons, because a rule excluded
everywhere is a rule you have not adopted.
The most useful thing that came out of it: gocritic's ifElseChain suggestion is
NOT safe to apply mechanically. Rewriting if/else to switch changes break
semantics — the break that exits the loop starts exiting the switch. That turned
a termination condition into an infinite loop in merge.go, and item.go's
integration loop had three loop-breaking breaks inside a chain it wanted
rewritten. Labelled breaks now, and the hazard is written down.
Documentation defects found the same way: a 25-line block documenting
decodeAwarenessEntries sat above ParseAwarenessStateJSON; YArray.Splice and
YXmlFragment.Slice both carried ToArray's description; ~70 exported comments did
not name the identifier they document, which is what godoc renders.
Also fixed: protocol.InspectMessage wrapped an inner error with %v instead of %w,
so it could not be unwrapped.
New: SECURITY.md, CONTRIBUTING.md, CHANGELOG.md. SECURITY.md states the threat
model plainly — this library decodes untrusted bytes, so anything reachable from
the decoders that panics, hangs or allocates without bound is a vulnerability
here rather than caller error — and records that the panic audit was a trace
through the guards, not "the fuzzer has not found one".
README corrected: it claimed Go 1.24+ while go.mod requires 1.26, and described
implementing persistence.Store as "the whole job" although the checkpoint profile
has existed since v0.0.2.
v0.0.6 — concurrency is part of the persistence contract, and single-codec stores are first-class
BREAKING, and deliberately so. Four exported conformance functions added on main
after v0.0.5 — PersistenceConcurrency, PersistenceCompactionConcurrency,
PersistenceFencingConcurrency, CheckpointPersistenceConcurrency — are removed.
They were never in a tag. Concurrency is not an optional capability of a
persistence store, and a suite a consumer could omit would let a store claim
conformance while skipping the rules hardest to satisfy. The behaviour now runs
inside Persistence, PersistenceCompaction, PersistenceFencing,
CheckpointPersistence and CheckpointPersistenceFencing, with no opt-out.
THE FIX MOST LIKELY TO AFFECT YOU. CheckpointPersistenceDeletion and
CheckpointPersistenceDeletionFencing hardcoded EncodingV1 in every seed, so a
store supporting only one codec failed them by construction — while
CheckpointPersistence in the same package carried an ErrUnsupportedEncoding skip
specifically to accommodate such a store. Two suites disagreeing about whether a
single-codec store is legitimate. Both now seed through the codec probe. If you
kept a codec alive only to satisfy those fixtures, you can delete it.
That survived because nothing in this repository had ever taken the shape. Every
fixture accepted both codecs and stored the vector it was handed, so the skip,
its guard, and the contract's permission to READ a checkpoint were carried with
no implementation to exercise them — correct, reviewed, and inert. There is now a
bare-update fixture: one codec, vector derived on load, run against every
checkpoint suite. It found two further defects within minutes of existing.
CONCURRENCY. The contract's concurrent clauses had no concurrent tests. Compactor
promises to preserve "concurrent appends after Basis" and its only test ran
entirely in program order. Store and CheckpointStore never said their methods
could be called concurrently at all — a Go type is not goroutine-safe unless it
says so — so the new suites were briefly imposing an interface stronger than the
one documented. Both profiles now state it: methods are safe for concurrent
invocation across documents, per-document revision, fence and replace decisions
are atomic, and a checkpoint is ONE value.
Certified: concurrent appends are durable with distinct increasing revisions,
appends racing a compaction survive, fence authority is decided by a single order
so a superseded owner never lands a write after its successor, and a load never
returns one save's update beside another save's vector.
Eight planted defects prove the suites reject rather than merely accept, and each
must fail AT ITS OWN ASSERTION — requiring only that the child failed repeats the
defect this whole exercise is about.
CONTRACT. Deleter's load-after-delete clause contradicted ErrNotFound's own rule
for any store whose delete leaves a pointer owned by another system dangling.
ErrNotFound wins, because reporting it there makes a caller treat a document that
HAD content as new and seed it with create-time content. Deleter now states that
its clause assumes the implementation owns everything that makes the document
findable; a partial owner's guarantee belongs to whatever owns the whole
document.
v0.0.5 — the checkpoint codec is declared, and initialization belongs to the generation
Two fixes, both for defects that were silent by construction: each had a failure
mode that produced no error and no failing test.
THE CHECKPOINT CODEC (#11). SaveCheckpointRequest gained an Encoding field
(EncodingV1 / EncodingV2), Checkpoint carries it back, and two sentinels were
added: ErrEncodingRequired and ErrUnsupportedEncoding. EncodingUnspecified is
never valid.
Why it could not stay optional: a decoder handed bytes in the wrong codec returns
an EMPTY document with NO error, in both directions. That is measured, not
assumed. A store that infers the codec therefore cannot distinguish "wrong codec"
from "empty document", so the data-loss path and the ordinary path look the same
from the inside. The writer now declares the encoding and the reader never
guesses.
INITIALIZATION LIFETIME (#12). Registry.Acquire ran the OpenFunc on the creating
caller's goroutine under the creating caller's context, which made an arbitrary
participant the owner of work everyone else depended on: coalesce a 2s caller and
a 60s caller, and if the 2s caller wins the race the 60s caller's load dies at 2s
for reasons it cannot see. Initialization now belongs to the generation, on its
own goroutine under a context this registry owns.
The generation context is WithCancel(WithoutCancel(callerCtx)). It keeps the
winning caller's VALUES — tenant scope, credentials, trace context, which the
OpenFunc was written against — and drops only its lifetime. It is cancelled when
the last waiter leaves, on Invalidate, and on Close.
CONTRACT CHANGE FOR CONSUMERS. OpenFunc no longer inherits the acquirer's
deadline. Anything relying on that must impose its own bound, derived from the
given context so the values survive. Note the part that is easy to miss: new
arrivals RENEW the last-waiter condition, so a document under load can hold the
waiter count above zero indefinitely and a wedged store is then never bounded by
this context at all — which is precisely the case where it is wedged.
Three further defects were found while reviewing that change, all reachable:
- A cancelled generation stayed joinable, so the next caller attached to an
initialization that could no longer succeed. - A failed open closed its completion channel twice and panicked. Caught by
this registry's own conformance suite, not by the new tests. - A waiter could be handed a document Evict or Close had already destroyed —
two callers then mutating different documents under one ID.
Invalidate now also waits for an abandoned initialization of the same document,
so after it returns no load of that document is in flight, including one no
caller is waiting for.
Every one of the eleven new registry tests is verified against a planted defect.
Two first-draft tests were hollow and only injection revealed it; one was a race
dressed as a barrier and is now genuinely ordered.
v0.0.4 — fenced checkpoint deletion, and a sentinel that pointed at data loss
Two new exported identifiers, nothing removed, nothing changed. No behaviour change to any existing implementation.
go get github.com/antst/go-yjs@v0.0.4
The coverage gap
PersistenceDeletionFencing covered the log profile from the start and nothing covered the checkpoint profile — an oversight in v0.0.3, not a decision. A checkpoint store could pass every shipped suite while letting a superseded owner erase the document its replacement was serving.
CheckpointPersistenceDeletionFencing asserts the ordinary rejection pair, plus the two that get missed:
- Both rejections must leave the state intact. Returning the right error is not enough — validating the fence and removing state in the same block passes an error-only check with the damage already done. That is what the reporter wrote on their first attempt, before their own test caught it.
- Deletion must not reset the fence high-water mark. A store that forgets the epoch when it forgets the state lets a superseded owner write again at exactly the moment a cascade is running and a stale node is most likely still trying.
Twelve deliberately broken stores now run against the conformance suites in child processes, each of which must fail.
The documentation defect — the more valuable half
An implementer resolving documents through a pointer asked which sentinel a resolved-but-missing target should use.
| said | so a literal reader concludes | |
|---|---|---|
ErrCorrupt |
"durable bytes that cannot form the history" | does not apply — there are no bytes |
ErrNotFound |
"no durable history exists" | this matches |
The wording steered toward the answer that loses data. ErrNotFound makes the caller treat a document that had content as new, seed it with create-time content, and overwrite the last good state on the next save — silently, arriving through the error type.
ErrCorrupt now covers state that is referenced but absent, not only malformed bytes. ErrNotFound now states that it means the store holds nothing — never saved, or deleted — and must not be used for state the store knows about and cannot produce.
Status
The backend layer has now been through a complete real port. Four contract gaps were found and closed, and then the same port completed its remaining work without finding a fifth — which distinguishes an incomplete contract from one nobody was still testing.
crdt/ and protocol/ remain ready to build on: differential oracle across 13 surfaces in both directions on every push, both codecs byte-exact, 20 fuzz targets over the untrusted-byte surface.
It has still not run for a week.
v0.0.3 — deletion, and the erasure ordering trap
Purely additive over v0.0.2: 13 new exported identifiers, nothing removed and nothing changed. Existing code compiles untouched.
go get github.com/antst/go-yjs@v0.0.3
What is new
backend/persistence gains a Deleter, because it had no way to remove a document's durable state at all — not on either profile, not anywhere. An application holding only the contract could create and replace forever and never erase, so every deployment had to reach around the contract to its own backend. Owner-delete cascades, tenant deletion and erasure requests are ordinary requirements of anything storing user content.
type Deleter interface {
Delete(context.Context, DeleteRequest) error
}DeletingStore and DeletingCheckpointStore compose it with each profile, mirroring CompactingStore.
Optional rather than mandatory
The obvious objection: compaction is a performance strategy, deletion is compliance, so Delete belongs on the store interfaces directly.
It loses to a case that objection does not consider — some media are forbidden to delete. WORM storage, object locks, regulated archival. The same compliance pressure demanding erasure in one regime demands immutability in another. A caller needing erasure type-asserts and fails loudly when it is absent, which beats an implementation whose Delete silently does nothing.
Semantics
- Idempotent — deleting an absent document returns nil, never
ErrNotFound. A cascade retries; the second attempt must not fail the operation it is completing. - Strict, not eventual — after
Deletereturns nil, a subsequent load MUST reportErrNotFound. A load still returning content is indistinguishable from the delete not happening. - Fence-aware, with a rejected delete leaving state intact — a superseded owner must not erase what its replacement is serving.
- Context-honouring.
The ordering trap
This existed before this release and applied to anyone deleting out of band.
Deleting durable state while a document is live resurrects it: the live document is unaffected and its next save writes the state back. Registry.Invalidate is necessary and not sufficient — it drains the current generation, and a later Acquire opens a fresh one from storage that has not been deleted yet.
Erasure needs three steps: stop admitting acquisitions (application state the OpenFunc consults), then Invalidate, then delete. Getting it wrong loses no data and reports no error; it silently restores content that was supposed to be erased. Documented on both Deleter and InProcessRegistry.Invalidate.
Tests
Eleven subtests across three suites, including that a document can be written again after deletion without its old history returning.
Ten deliberately broken stores now run against the conformance suites in child processes and each must fail — six for the checkpoint contract, four for deletion (non-idempotent, leaves state, purges everything, ignores cancellation). The rejection test fails if either violation list is empty.
Status
crdt/ and protocol/ remain ready to build on — differential oracle across 13 surfaces in both directions on every push, both codecs byte-exact, 20 fuzz targets over the untrusted-byte surface.
The backend layer has now had four contract gaps found by a single real port, in two days. That is the argument both for shipping conformance suites and for continuing to treat the layer as young. Nothing here has run for a week.
v0.0.2 — checkpoint persistence profile
Purely additive over v0.0.1: 13 new exported identifiers, nothing removed and nothing changed. Existing code compiles untouched.
go get github.com/antst/go-yjs@v0.0.2
What is new
backend/persistence gains a second profile.
Until now the contract had exactly one shape — an append log — and conformance.Persistence enforced it by appending two opaque byte strings and requiring both back, in order, byte-identical. That is right for a medium that can hold an ordered log of independently durable records. It is impossible for one whose durable unit is a document-sized blob rewritten in place: an object-store key, a file whose id is a stable pointer, a single row.
CheckpointStore serves that medium:
type CheckpointStore interface {
SaveCheckpoint(context.Context, SaveCheckpointRequest) (Revision, error)
LoadCheckpoint(context.Context, backend.DocumentID) (Checkpoint, error)
FenceMode() FenceMode
}Same sentinels, same caller-owned-slice discipline, same returning-nil-means-durable rule, FenceMode fixed at construction. Methods are name-qualified so one type can offer both profiles.
conformance.CheckpointPersistence and CheckpointPersistenceFencing assert what is meaningful for the shape — round-trip fidelity, replacement, monotonic revisions, byte ownership in both directions, document isolation, explicit absence, cancellation, fencing — and deliberately assert no per-record history or pagination. Six deliberately broken stores are run against the suite in child processes and each must fail, so it is proven to reject rather than only to accept.
Example_checkpointBackend is a worked end-to-end flow in exactly that shape: one rewritable blob, no envelope, state vector derived on read, persisting from the update seam and recovering cold from storage alone. It is held to the public suite and verified to fail when persistence is disabled.
Why it exists
A service porting to this library could not express its real backend. The durable form is one blob per document, and that blob is a bare Yjs V2 state another repository also writes — so it can carry no envelope. The three ways to force it through the log suite were: one blob per record (abandoning the stable pointer the medium exists to provide), a log framed inside the blob (breaking the cross-repo format), or skipping the suite for the implementation that actually runs in production. All three are worse than adding the profile.
Also corrected
Both found by that same port rather than by us:
Coordinatorclaimed persistence provides the final stale-owner rejection. True only of a fenced store — and the checkpoint profile makes unfenced ordinary. An implementer would have believed they had defence in depth while having one layer. The doc now says the backstop is conditional, and that a node must shed clients on lease loss rather than merely decline to write.- The checkpoint suite fed opaque bytes, failing a store for deriving its state vector on read — a strategy the contract explicitly permits. Fixtures are now genuine Yjs updates;
StateVectorbecame caller-required so no store is forced to parse CRDT bytes.
Status is unchanged from v0.0.1
crdt/ and protocol/ are ready to build on — differential oracle across 13 surfaces in both directions on every push, both codecs byte-exact, 20 fuzz targets over the untrusted-byte surface.
The backend layer is still young. It now has one real implementation in progress against it, which is one more than at v0.0.1 and still not a production hour. Nothing here has run for a week.
v0.0.1 — first release
A Go implementation of the Yjs CRDT, wire-compatible with the JavaScript reference, for backends that need to serve Yjs to browsers.
go get github.com/antst/go-yjs@v0.0.1
Requires Go 1.24+. No runtime dependencies.
Version numbering starts fresh
This began as a fork of skyterra/y-crdt, whose copyright is retained in LICENSE — but it is a different product now. The V2 codec, the sync and awareness protocol package, the backend ports and their conformance suites, the differential oracle and the fuzz targets were all added here; 60 Go files became 260. Continuing that version line would imply a continuity that does not exist.
What is proven
Compatibility with yjs@13.6.31 is enforced by a differential oracle, not by hand-written expectations: random operation sequences run through both implementations and the results are compared byte for byte across 13 surfaces, in both directions, on every push — 20,000 seeds per push, tiers to 10,000,000.
Direction B is the half that matters most: only bytes produced here and decoded by the reference can catch a non-canonical encoding.
- Both V1 and V2 update codecs complete and byte-exact, including delta-coded delete sets
- Twenty native fuzz targets over every entry point accepting untrusted bytes — no panic, no hang, allocation bounded by input length
- Whole-module race detector, lint, and vet in both build configurations gate every push
- Faster than yjs, yrs and ygo on 49 of 54 matched benchmarks in a bracket-certified run
Why 0.0.1 and not 1.0
The CRDT core is well evidenced. The backend layer is not.
backend/{persistence,cluster,memory,hub,conformance}, Registry.Invalidate and Handle.Done were written days before this tag. They have conformance suites and a worked single-process example that runs in CI, and they have never served a real client. Nothing here has run for an hour, while a collaboration server runs for weeks — elapsed time is the dimension with no evidence behind it.
The API is also still moving: NewDoc changed shape and ToDOM was removed in the week of this release.
Treat crdt/ and protocol/ as ready to build on, and backend/ as code you are co-developing. Run your own store against backend/conformance before trusting it, and get a long-running instance up early.
Layout
| Package | What it is |
|---|---|
crdt |
Documents, shared types, update encoding, transactions, snapshots, undo |
protocol |
Sync and awareness message framing |
backend/persistence |
Port you implement — SQL, files, object store |
backend/memory, backend/hub |
Shipped in-process defaults, replaceable |
backend/cluster |
Optional. A single process is first-class |
backend/conformance |
Public suites to check your implementation against |