Benchmarks

Benchmarks

These numbers are relative guidance from a single machine (16 logical CPUs, Java 25, warm JVM), not absolute figures. GC, the OS file cache, and JIT move them run to run — the shape is what matters. Treat in-memory throughput as order-of-magnitude.

The figures come from the config.stress.* suite (AbstractStressTest + one subclass per codec). It is disabled by default — a normal ./gradlew test skips it. Run it with:

./gradlew test -Pstress

-Pstress sets -Deveryconfig.stress=true and raises the test heap to 2 GB. Each codec writes a detailed report to build/stress-report/<codec>.md. Tune the load with the memConfigs() / threads() / scaleEntities() / parallelMillis() hooks on AbstractStressTest. The run below used 100 000 entities, 16 threads, and 1.5 s measurement windows.

The one-line story

EveryConfig shares a single thread-safe Jackson ObjectMapper per codec across every live Config. So a config costs only its tree (~1.9 KB), and you scale by opening many configs across threads — which the benchmark sustains at 1–1.7 million binds/second with zero corruption and no deadlock. A single Config is a single-writer handle (see Lock cost).

Single-thread throughput

Ops per second, higher is better. The in-memory dynamic API never touches the format, so those rows are codec-independent (the spread is JIT/GC noise, not a real difference). Binding uses the shared mapper; save is dominated by the atomic file write.

Operation	Rate (per second)
setValue / getInt / getOrSetValueIfAbsent (in-memory)	~2–10 million
bind write — setValue(pojo)	~0.3–0.4 million
bind read — getValue(type)	~0.2–0.8 million
save (encode + atomic write + .bak + fingerprint)	~1 100–1 300
save + reload round-trip	~850–980

The takeaway: the dynamic tree and binding are far faster than any realistic config workload; the durable save is I/O-bound at ~1 k/s (atomic temp-file write + rename + backup + fingerprint), not engine-bound.

Scale — one config with 100 000 entities

200 000 keys in a single config (entities.eN.name + entities.eN.value), built, saved, and reopened.

Metric	JSON	YAML	TOML	JSONC
Build (200k setValue)	78 ms	47 ms	94 ms	76 ms
Save (encode + write)	101 ms	292 ms	301 ms	144 ms
On-disk size	7.5 MB	4.5 MB	4.9 MB	6.8 MB
Reopen + parse	105 ms	594 ms	375 ms	260 ms
getKeys(deep) over 300k paths	42 ms	22 ms	48 ms	54 ms

A 100 000-entity config builds, saves and reloads in well under a second on every codec. (Getting here fixed two real bugs — see Findings & fixes.)

Concurrency — 16 threads

The supported model is many configs over one shared codec. Every scenario below recorded 0 integrity mismatches, 0 round-trip mismatches, and no deadlock or hang.

Scenario (16 threads, 1.5 s)	JSON	YAML	TOML	JSONC
Distinct configs — set → save → reload → verify (per second)	~2 100	~2 100	~2 000	~2 100
Shared mapper — concurrent setValue(pojo) + bind back (per second)	~1.72 M	~1.19 M	~1.12 M	~1.58 M
One config, externally synchronized (per second)	~77 k	~75 k	~56 k	~67 k

The shared mapper takes over a million concurrent binds/second — Jackson's documented thread-safety holds, and the per-codec mapper is never the bottleneck.

The cost of a lock

We deliberately do not make a single Config internally thread-safe. The benchmark measures why.

Measurement (16 threads)	JSON	YAML	TOML	JSONC
Uncontended lock tax (lock/unlock per setValue)	~8 ns	~3 ns	~0 ns	~7 ns
Parallel — a Config per thread, lock-free (ops/s)	~77 M	~77 M	~66 M	~74 M
Serialized — one shared Config, one lock (ops/s)	~8.4 M	~10.4 M	~8.0 M	~9.9 M
Throughput forfeited by serializing	9.2×	7.3×	8.3×	7.5×

The per-call lock itself is nearly free (single-digit nanoseconds). The real cost is serialization: funnelling every operation through one lock to make a single config safe throws away ~8× of throughput — the exact parallelism the shared-mapper design exists to provide. So the contract is:

A Config is a single-writer handle. Scale with many configs across threads (fully supported above). If you must share one config across threads, put it behind your own lock — a synchronized block over your setValue/save calls is enough (the "externally synchronized" row proves it: zero errors).

Memory

Metric	Value (all codecs)
Retained per live Config	~1.9 KB
20 000 live configs	~36 MB
Distinct ObjectMapper instances	1 per codec

A config's footprint is its tree, not an engine. All 20 000 configs returned the same mapper identity — there is no per-file engine, pool, or ThreadLocal.

What the numbers say

• Open as many configs as you like. They are cheap (~1.9 KB) and share one mapper; concurrency scales with distinct configs, not with a heavier engine.
• Saves are I/O-bound (~1 k/s). If you need more, prefer saveIfDirty(), batch your mutations before saving, and don't reload() right after your own save() (the in-memory tree is already authoritative).
• One config is single-writer. Confine it to a thread or guard it yourself; full internal locking would cost ~8× and is not worth it for a case external synchronization handles cleanly.

Findings & fixes

The 100k run surfaced two real defects, both now fixed:

1. O(n²) in the comment-aware emitter. The key-order reconciliation used ArrayList.contains per key, so a node with many siblings was quadratic. Saving 100k entities took ~14–15 s on YAML/TOML/JSONC. Switching the membership check to a LinkedHashSet made it O(n): the same save is now ~150–300 ms (50–100× faster). JSON was never affected (it serializes through Jackson directly).
2. YAML's 3 MB input cap. SnakeYAML's default codePointLimit rejected a ~4.5 MB YAML file, so a large config failed to reload (and backed up to .bak). The YAML codec now raises the limit (config files are trusted local data), and the 100k config round-trips cleanly.

One characterized, by-design limitation remains: concurrent unsynchronized mutation of a single config can throw ConcurrentModificationException (an intermittent race — save()'s emit iterating the tree while another thread mutates it). save() now emits from a snapshot taken under its lock, which closes the long emit window; the residual race is the brief copy itself, and the supported fix is single-writer use or external synchronization (above).

Caveats

• Single machine, single run, warm JVM — relative guidance only.
• In-memory throughput is JIT/GC-sensitive; the per-codec spread there is noise.
• save/reload numbers include real filesystem I/O (atomic write, .bak, fingerprint), so they reflect your disk as much as the library.
• The concurrency tests assert correctness (0 mismatches, no deadlock); the throughput they print is secondary.

Reproduce

./gradlew test -Pstress                     # all codecs, default load
# tune by overriding the hooks in a subclass, or read build/stress-report/<codec>.md

See also Lifecycle, Reload & Watching for the save/reload contract and Architecture Overview for why the mapper is shared.