Evidence

Evidence

Quality in this package is checked by tests that exercise every outcome of the sync decision, and by a command-line session that anyone can repeat. Nothing here measures the benefit the design promises, a saving in tokens, because no measurement has been taken; the evidence covers correctness of the mechanism and says nothing about its payoff.

Tests

The suite holds 50 tests in 10 files. All 50 passed on the run recorded for this wiki (Python 3.12.10 on Windows), in 0.75 seconds, with no network access and no API key.

File	Tests	What it checks
test_sync.py	8	Bootstrap from the fields store; a second sync is a no-op; a packet edit reaches the Markdown; an edit to the Markdown fields comment reaches the packet; an edit to the notes region resyncs and keeps the fields; both one-way syncs are idempotent; editing both sides raises a conflict
test_human_doc.py	8	Markers and bullets render; the fields comment, schema comment, and notes each round-trip; the notes region is editable independently of the fields; each missing marker raises
test_schema.py	8	Valid fields pass; a missing required field and a bad enum value raise; numeric strings from a packet coerce and pass; a genuinely invalid number string still raises; native types are unaffected; the caller's dictionary is not mutated
test_cpn.py	7	Keys sort with SCHEMA_VER first; floats format to two decimals; output is deterministic and ends in one newline; parsing recovers the version and fields, ignores blank lines, and raises without SCHEMA_VER
test_cli.py	5	sync bootstraps; compile is an alias; validate reports valid and invalid; expand prints without writing
test_fields.py	4	Load, save, round trip, the merged SCHEMA_VER view, and LF line endings
test_loader.py	4	A valid packet loads; the Markdown is used when the packet is missing or fails the schema; both missing raises
test_hashing.py	3	Equal text gives equal hashes, different text differs, and the empty string matches the known SHA-256
test_lockfile.py	2	A missing lock gives empty hashes, and save then load round-trips
test_end_to_end.py	1	Bootstrap, a runtime load, a deliberately corrupted packet, and the fallback to the Markdown

The end-to-end test is the most informative single check. It builds an entity_resolution artifact with a float field, bootstraps it, loads the packet and finds ID_THRESHOLD=0.92 in it, corrupts the TASK value to one that violates the schema's enum, and asserts that the loader then returns the Markdown file's text. The comments in the test explain that only one field is broken, so that the failure isolates the enum check and not the required-field check.

A session you can repeat

These results come from a run on the example artifact, in a scratch copy, for this wiki.

Step	Output
rosetta validate rosetta_build --dir <copy>	valid
rosetta sync on the unmodified copy	sync: no-op
Change DECISION_COUNT=3 to 4 in the packet, rosetta sync	sync: machine-to-human
rosetta sync again	sync: no-op
Change the packet to 5 and the Markdown comment and bullet to 9, rosetta sync	CONFLICT: conflicting fields: DECISION_COUNT, exit code 1
Restore the Markdown to its last synced content, rosetta sync	sync: machine-to-human, exit code 0, packet and fields now read 5
Tighten the schema minimum to 10, rosetta validate	INVALID: DECISION_COUNT: 4 is less than the minimum of 10, exit code 1

Check it yourself in 5 minutes

git clone https://github.com/dboles99/rosetta.git
cd rosetta
python -m venv .venv
.venv/bin/pip install -e ".[dev]"      # Windows: .venv\Scripts\pip install -e ".[dev]"
.venv/bin/python -m pytest -q
cp -r examples/rosetta_build /tmp/ex
.venv/bin/rosetta validate rosetta_build --dir /tmp/ex
sed -i 's/^DECISION_COUNT=3/DECISION_COUNT=4/' /tmp/ex/rosetta_build.cpn
.venv/bin/rosetta sync rosetta_build --dir /tmp/ex
.venv/bin/rosetta sync rosetta_build --dir /tmp/ex

Expected output: 50 passed, then valid, then sync: machine-to-human, then sync: no-op. To see the limit, copy the example afresh, change "DECISION_COUNT": 3 to 7 in rosetta_build.fields.json only, and run sync: the output is sync: no-op and the packet still reads DECISION_COUNT=3.

Known failures

Each item below was reproduced in a scratch copy, and the first two are stated in the README as well.

• An edit to fields.json alone is not picked up. Sync compares the two views with the lock and never the fields file.
• After a packet sync, numbers become strings in fields.json. The integer 3 becomes the string "4"; validate coerces it back, though a reader of the JSON sees a change of type.
• A conflict cannot be cleared by making the views agree. Detection compares each view with the lock, so a Markdown file edited to match the packet still reports CONFLICT: conflicting fields: with an empty list. Restoring one view to its locked content clears it.
• The Markdown bullets are display only. Editing a bullet and nothing else makes sync report sync: human-to-machine, yet the packet and the fields file keep the old value, because only the ROSETTA:FIELDS comment is read.
• The packet rounds floats to two decimals. A field of 0.925 appears as RATE=0.93 in the packet and as 0.925 in the Markdown from the first bootstrap.
• Token savings are unmeasured. The packet is shorter than the document in the example, which anyone can see; whether that changes cost or quality has not been tested.