Design and Internals

Design & Internals

What this page covers: the design rationale behind the load-bearing parts of the library, plus the two cross-cutting idioms that explain most of the codebase once you've seen them — capabilities are interfaces, not flags and there is no global registry. This is the page to read before changing something that "looks redundant" — it usually isn't. The wiki teaches; the Javadoc specifies.

📌 Note — this page is the canonical home for why a design is the way it is (and the hazard analysis behind it). The Javadoc owns the exact per-member contract; the in-repo READMEs (README.md, manager/README.md) carry the practical narrative next to the code.

---

The rationale map

Each slice of the library has a self-contained decision behind it. The rationale lives next to the implementation it governs — the project README.md, the manager/README.md for the refs/caching add-on, and the extensive Javadoc — and the highlights are summarised here.

Subsystem	Status	Why it's built this way
Refs & managers (manager/, manager/README.md)	Implemented	The everydatabase-manager add-on: why refs and caching are a separate façade module, the three decoupled concerns (serialization / type recovery / resolution), the cell + monotonic-stamp + tombstone model, and the per-context RefRegistry (no global registry).
Distribution modules (core/build.gradle, libby/build.gradle)	Binding	The two distribution flavors (core / libby): the dependency-scope decisions, the GPL MySQL-driver handling (POM metadata in core, runtime download in libby — never bundled), the slf4j compileOnly decision, and why libby's POM excludes core's transitive set.
Storage logging (log/ package)	Binding	The logging subsystem: the silent-by-default policy with an unsuppressible ERROR floor, the topic/level model, the privacy defaults (counts/durations, never content), the sink-resolution order, and the implementation checklist of which ops emit.
Cache sync (changefeed, manager.sync, specs/SPEC_cache_sync.md)	Implemented	Cross-process cache invalidation: why a capability (ChangeFeedStorage) not a flag, the push (Mongo Change Streams / Postgres LISTEN/NOTIFY) vs pull (PollingCacheSync over Repository.versions) split per backend, why a late/duplicate/dropped event is safe by construction (the cell stamp guard), and why the consumer lives in :manager (where the cache is) while the capability lives in :core.
This wiki	Proposed	This wiki itself: taxonomy, page inventory, the source-of-truth hierarchy (README sells, wiki teaches, Javadoc specifies), and the callout/code/language conventions every page follows.

📌 Note — read each rationale next to the implementation it governs: the distribution decisions next to core/build.gradle and libby/build.gradle, the logging decisions next to the log/ package, and the refs/caching rationale next to manager/README.md.

---

Idiom 1 — capabilities are interfaces, not flags

The central design idiom. Optional features are expressed as extra interfaces a Storage may implement, discovered with instanceof — never as a boolean you have to trust. The compiler stops you from using a capability a backend doesn't have.

if (storage instanceof TransactionalStorage tx) {
    tx.inTransaction(scope -> { /* ... */ });   // only reachable on backends that actually support it
}
if (storage instanceof SchemaAwareStorage schema) {
    schema.register(new V001_Init()).migrate().join();
}

The capability interfaces:

Interface	Adds	Backends that implement it
tx.TransactionalStorage	inTransaction(scope -> future)	SQL (incl. H2), Mongo (replica set), InMemory (no isolation) — not LocalFile
schema.SchemaAwareStorage	register(...).migrate(), forward-only, _schema_migrations	SQL, Mongo, LocalFile — not InMemory
changefeed.ChangeFeedStorage	subscribe(listener) + originId(), a push feed for cross-process cache invalidation	Mongo (Change Streams), PostgreSQL (LISTEN/NOTIFY), InMemory — not MySQL/MariaDB, H2, LocalFile (they poll)

🧭 Decision — a flag (storage.supportsTransactions()) would let a backend claim a feature it can't honor, deferring the failure to runtime. An interface makes "no backend pretends to support something it can't" a compile-time fact. Preserve this idiom when adding features: a new optional capability is a new interface a subset of backends implements, not a method on Storage that throws UnsupportedOperationException. See Architecture Overview, Transactions, Schema Migrations.

A related expression of the same principle: optimistic locking is enforced only by the backends that genuinely can (MySQL/MariaDB, PostgreSQL, Mongo). H2 opts out explicitly (H2SqlRepository.supportsVersioning() returns false) and silently degrades to plain upsert — documented and tested, never a silent lie. See Optimistic Locking.

---

Idiom 2 — there is no global registry (manager module)

The manager add-on used to be tempted toward a process-wide static Class<?> → resolver map. It deliberately doesn't have one. A RefRegistry is a per-context instance (new RefRegistry()), and a Ref resolves only against the registry it was bound to.

RefRegistry survival = new RefRegistry();
RefRegistry lobby    = new RefRegistry();
survival.manager(heroDesc, survivalStore, CachePolicy.always());  // Hero -> survivalStore
lobby.manager(heroDesc,    lobbyStore,    CachePolicy.always());  // Hero -> lobbyStore — no collision

🧭 Decision — a static default would let two independent contexts (two plugins) register a manager for the same entity type and silently collide (last-writer-wins), discovered only in production. Making the registry per-context turns that silent collision into a thing that can't happen: two registries can each own a Hero manager on different storages. The registry vends what a ref-aware setup needs (codec(Type.class), manager(...), ref(key, Type.class)) so you can't half-wire it. A bare Ref.of(key, type, null) (the explicit unbound form, with a null registry) fails fast on resolve — never resolves against "some default". See Caching & References.

This is the same instinct as Idiom 1: make the failure mode impossible by construction rather than something you document and hope nobody hits.

---

Other internals worth knowing

A few load-bearing decisions that surprise people reading the source:

• Everything is async; there are no blocking variants. Every I/O method returns CompletableFuture; callers use .join() or compose. This isn't an oversight — a sync overload would double the surface and invite blocking on the wrong thread. See The Async API.
• Keys persist by toString() (≤ 255 chars), match by equals/hashCode. The 255 cap is the safe intersection across all backends; an oversized key makes save complete exceptionally rather than silently truncate into a collision. See Entities, Keys & Collections.
• Collection names are validated against ^[a-zA-Z][a-zA-Z0-9_]*$ — the intersection of identifier rules across every backend, so no quoting is ever needed.
• The cache cell/stamp/tombstone model (manager module) makes concurrent writes and reloads safe: writes swap the value in place under a monotonic stamp so a slow reload can't regress a newer write nor resurrect a newer delete. These internals now live in the manager.cache package and are open for extension — the classes (CacheEntry, CacheOptions, CachePolicy, LruCacheStore) are non-final with protected members, and CachingManager itself is subclassable — so a deployment can add metrics or swap the backing map. Most code configures via CacheOptions/CachePolicy and never subclasses. Detailed in manager/README.md; surfaced in Caching Managers and Extending the Library.
• The dual-target build. Production code is authored in Java 17 syntax but compiled to Java 8 bytecode via Jabel (--release 8), while the toolchain that runs the tests is JDK 25. So production code must stay Java-8-runtime-safe (no Java 9+ APIs) even though it reads like modern Java — e.g. StorageExecutors reflects newVirtualThreadPerTaskExecutor instead of calling it directly. See Building from Source.
• Logging is silent by default with an ERROR floor. Routine ops emit nothing under the factory default; failures always emit, and no configuration can switch that floor off. Privacy is the default: events carry counts/durations/collection names, never entity content. Rationale in the log/ package Javadoc; usage in Logging & Diagnostics.

---

The source-of-truth hierarchy

When you're unsure which document to trust, this is the order:

README sells and starts · the wiki teaches · the Javadoc specifies.

A fact lives in exactly one canonical place; everywhere else links to it. Version numbers live in the version catalog and are surfaced on a single page — Dependency Versions & Overrides — and never restated elsewhere. Wiki code examples mirror tested entity shapes (TestPlayer, PlayerProfile, …) so they can't silently rot.

---

See also

• Architecture Overview — the Storage / Repository / EntityDescriptor / Codec / Storages surface and the capability idiom in context.
• The Async API — the CompletableFuture model that pervades the whole library.
• Distribution Flavors — the flavor and GPL/MySQL decisions and their rationale.
• Logging & Diagnostics — the logging subsystem and its rationale.
• Caching & References — the everydatabase-manager add-on and its cell/stamp model.
• Cross-Process Cache Sync — the change-feed capability and the push/poll consumer.
• Extending the Library — the seams that are meant to be extended.
• Editing this Wiki — the conventions every page on this site follows.