Amendment Idea: Ephemeral Data Blobs #606
Platonenkov
started this conversation in
XLS Ideas (pre standard proposal)
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Abstract
A transaction commits to a payload without carrying it: it holds a hash and a size, while the
payload travels beside the transaction over the peer protocol, is served for a bounded number of
ledgers, and may then be dropped. Bulk data becomes priced by size and temporary by construction.
Unlike EIP-4844, availability is not a condition of validity at any point. Ethereum's consensus
nodes refuse a block until blob sidecars verify against their KZG commitments, which is what makes
"drop it after 18 days" safe there. Nothing here provides that guarantee, so this is a strictly
weaker primitive than the thing it borrows from, and the borrowing stops at "bounded retention".
1. Why not simply price memo bytes
Pricing memo bytes (XLS #413) is
necessary and we support it. What price cannot answer is duration: a memo is part of the
transaction, so whatever is written there is permanent for anyone keeping history, whatever it cost.
Raising the memo cap to ~1.3 MB makes that permanence the network's dominant storage term.
2. Mechanism
DataHashHash256DataSizeUInt32The payload is not part of the transaction and does not enter the transaction tree, the state tree
or the ledger hash. The signature commits to it through
DataHash, so it cannot be substituted.A server that has accepted a blob SHOULD serve it for
DataRetentionLedgersledgers and MAY drop itafterwards. A server MAY never have received it at all — because it started later, was offline, or
gossip did not reach it — so the API needs both
dataExpiredanddataNotFound, and callers musttreat the two as indistinguishable in the general case.
Transaction validity never depends on holding the payload: a server that has never seen it still
applies the transaction. This keeps availability out of consensus. It does not make the proposal
consensus-free — new fields, a new fee formula and new result codes are all amendment-gated.
2.1. Fee
DataSizeis priced per byte through fee-voted parameters, the way XLS-413 prices memo bytes.EscrowFinishis the in-protocol precedent for size-dependent pricing:base * (32 + size/16), about 0.625 drops per byte.Because the retention obligation is bounded rather than perpetual, a blob byte can cost less than a
permanent memo byte — which is the incentive to want, since it pulls bulk data out of memos.
3. Unsolved problems, stated plainly
These are not details. Any one of them may be fatal.
before it reaches a ledger and before any fee is burned. A transaction that dies in the queue, or
on
LastLedgerSequence, has already cost every peer that relayed it. This is a free amplifierand it needs relay rules of its own: accept a blob only alongside a transaction, verify
SHA-512Half(blob) == DataHashandlen == DataSizebefore forwarding, penalise peers that miss.volume per ledger. Bounding it means validators rejecting proposals that exceed it — a consensus
rule, not an API convention. Either the aggregate is unbounded or the "availability stays out of
consensus" framing does not extend to bandwidth.
DataSizemay not be enforceable at fee time. The fee is charged from a declared size, whilethe payload arrives out of band. Understating it must be caught at relay and made unprofitable;
the mechanism for that is not specified here.
See below — this is the hole that eats the use cases.
syncing inside the window with no incentive for peers to answer; deduplication when two
transactions share a
DataHash; whether resubmission restarts the window; whattx/account_txreturn and under what size limits; DoS surface of downloading before hash verification; hash
domain separation with rippled's
HashPrefix.requirements for every node. Fee voting has always set prices and reserves, not machine specs.
4. The case against this document
The strongest argument against this proposal comes from our own measurements.
The memo cap already makes bulk storage expensive and slow: 12.64 XRP and over a million
transactions per gigabyte. A byte price on memos makes the cost of a gigabyte roughly independent of
where the cap sits. Live traffic shows nobody storing files in memos at all — the median memo is 84
bytes. XLS-413 with sensible constants, plus leaving the 1 KB cap alone, makes bulk storage
expensive enough that nobody bothers — with no new peer-protocol layer, at the cost of one amendment
that is being discussed anyway.
What it does not do — and cannot, because a price answers "how much" and never "for how long" — is
bound retention. Our own upload is the demonstration: 3.3 MB for four cents, permanent. Under
XLS-413's defaults it would have cost twenty cents, and still be permanent. While the cap is 1 KB
that difference is academic, because nobody will pay 65 XRP for a gigabyte of permanence. If the cap
rises, it is the only difference that matters.
And the availability hole empties the use cases: a receipt or an attachment nobody can fetch is
worthless, while proof-of-existence needs only 32 bytes of hash in a memo today, with no protocol
change at all. The honest question a reviewer will ask first is "how is this better than a hash in a
memo plus external storage?" — and the only candidate answer, that every peer saw the data at
inclusion time, is exactly what the design declines to guarantee.
So this belongs in a drawer unless the cap is raised. If it is raised, this is the shape we would
argue for, because bounded retention is the only lever left once you accept that no XRPL mechanism
pays operators to store anything.
5. Prior art
rippled#4259 (2022) — memo fee scaling, both closed
unmerged.
closed; the objection was that rate-based rules are incompatible with deterministic consensus.
A fee derived from
DataSizeinsidecalculateBaseFeeis a pure function of the transaction anddoes not inherit that problem.
6. Open questions
DataRetentionLedgersbe fixed by amendment rather than voted, so expectations stay stable?weaker than a KZG commitment make the whole construction pointless?
All reactions