Show and tell: deterministic storage/reconstruction benchmark — 99.26% reduction on 100-version repo history #19201
BamboozledLabs
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I built a deterministic storage/reconstruction system and tested it against 100 consecutive commits from the public psf/requests repository.
Measured result
Logical tracked-file history: 473,389,545 bytes
Required durable instance state: 3,459,434 bytes
Measured reduction: 99.2692204472%
Actual NTFS-allocated state: 3,469,312 bytes
Allocation-based reduction: 99.2671337936%
Exact restoration
100/100 snapshots restored
13,002/13,002 path occurrences independently verified
Relative path: PASS
Git mode: PASS
Length: PASS
SHA-256: PASS
2/2 corrected clean replications produced identical durable artifacts
4/4 applicable durable-component corruption attacks failed closed
Anti-cheat / accounting checks
The verification audit included:
same denominator across measured percentage controls
required durable basis accounted for
return map/index/witness accounted for
integrity state accounted for
orphan and stray state audit
cache/database/journal/sidecar audit
hard-link and symlink audit
reparse-point audit
sparse-file audit
NTFS compression audit
alternate-data-stream audit
hidden-authority-access audit
process/dependency audit
reconstruction-time network/socket audit
cold and warm reconstruction measurements
independent verifier separated from reconstruction code
Real defect found during testing
A single-bit mutation in previously non-enforced descriptor text initially did not fail closed.
I preserved the failure, corrected the integrity layer using the same storage method and terminal, charged the additional 95 durable bytes, and reran the complete clean builds, restores, independent verification, and corruption attacks.
Corrected result: PASS.
Conventional control
On the same denominator, a conventional exact content-addressed store plus losslessly compressed snapshot manifest required:
3,458,736 bytes
That is 698 bytes smaller than Hermione’s corrected representation on this particular population.
I am leaving that result in the receipts because I am not claiming universal superiority.
Scope
This is a highly related real-world version-history workload.
I am not claiming:
90%+ reduction for arbitrary unrelated data
superiority over every conventional storage system
a finished universal storage product
The conservative claim is:
Hermione Storage repeatedly demonstrated greater than 90% durable-storage reduction on the disclosed real-world test population while reconstructing every required artifact byte-for-byte under independent cryptographic verification.
The corrected measured occurrence was 99.2692204472%.
I’m keeping the proprietary reconstruction mechanism private.
What I’m looking for here is simple:
If you work on filesystems, snapshots, deduplication, content-addressed storage, compression, integrity, or reconstruction, attack the accounting and the verification.
If there is a cheat in the measurement, I want somebody here to find it.
If you want to discuss the receipts or an engineering evaluation privately, message me here on GitHub or use the contact listed on my profile.
All reactions