refactor(vara.eth/malachite-core): malachite block payloads versioning - #5506
Merged
grishasobol merged 13 commits intoJun 5, 2026
Merged
Conversation
grishasobol
added
type: refactor
grishasobol
force-pushed
the
gsobol/ethexe/malachite-core-concretize-payload
branch
from
May 25, 2026 10:54
733e281 to
eca538f
Compare
…generic
Replace the marker `BlockPayload` trait + blanket impl with a concrete
versioned struct `BlockPayload { version: u16, bytes: LimitedVec<u8, 1_024_000> }`,
and drop the `<P>` type parameter from `Block`, `Externalities`,
`MalachiteService`, `State`, `Store`, `BlockEntry`. The new
`BlockPayload` type and its constants (`BLOCK_PAYLOAD_VERSION`,
`MAX_BLOCK_PAYLOAD_BYTES`) live in `ethexe-common` so the executor
side can consume them without a back-dep on the consensus layer;
`ethexe-malachite-core` picks them up from there.
Application-level encoding/decoding of `ethexe_common::malachite::Transactions`
moves entirely to ethexe-malachite (the service crate) where it's
wrapped into `BlockPayload` before crossing the core boundary. The
1000 KiB byte cap keeps the encoded block under the 1 MiB engine
ceiling and turns oversize payloads into decode-time errors at the
wire.
Fixes #5505
grishasobol
force-pushed
the
gsobol/ethexe/malachite-core-concretize-payload
branch
from
May 25, 2026 17:15
fbd5c67 to
d3341a6
Compare
grishasobol
added
ai-generated
grishasobol
marked this pull request as ready for review
June 3, 2026 20:27
Contributor
|
Warning Gemini encountered an error creating the summary. You can try again by commenting |
…nsupported version Use DecodeAll for Transactions so a proposer can no longer pad BlockPayload::bytes with junk up to MAX_BLOCK_PAYLOAD_BYTES and still pass validation. Adds regressions for: - validate_block_above soft-rejecting an unsupported version, - validate_block_above soft-rejecting trailing garbage, - process_mb_proposal returning Err on the same version mismatch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…lockPayload Replace the versioned `BlockPayload` envelope with a raw, size-capped `LimitedVec<u8>` payload on the malachite `Block`, and move protocol versioning into the application schema itself. `Transaction` becomes `Operation`: a `#[repr(u32)]` append-only enum with a fixed-width tag and frozen discriminants. New protocol behaviour is an additive operation variant; `validate_block_above` gates which operations a fresh proposal may carry, while decode/execution stay permissive so finalized historical blocks always replay (sync, re-execution). The trailing-garbage and undecodable-payload paths are covered by tests. `MAX_BLOCK_PAYLOAD_BYTES` (1 MiB) caps the encoded block well under malachite's 4 MiB gossip `pubsub_max_size` — the whole block ships as a single proposal `Data` part and a finalized block syncs in one request-response round — with ample headroom over the ~127 KiB realistic content (`MAX_INJECTED_TRANSACTIONS_SIZE_PER_MB` plus the protocol bookend operations). `ethexe-malachite-core` no longer depends on `ethexe-common`: the payload is opaque bytes at the consensus boundary, restoring the application-agnostic layering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
grishasobol
force-pushed
the
gsobol/ethexe/malachite-core-concretize-payload
branch
from
June 4, 2026 15:54
6c10c2a to
bf67fa7
Compare
grishasobol
commented
Jun 4, 2026
grishasobol
changed the title
…ntly, reintroduce BlockPayload alias Apply the review-comment cleanups left over from the `Transaction`→`Operation` / `Transactions`→`Operations` rename: - `MbStorage` trait: `transactions` / `set_transactions` → `operations` / `set_operations`, and call sites updated to match. - `MbFullData.transactions` → `MbFullData.operations`. - Local variables (`transactions_hash`, `txs`, `dummy_txs`, `empty_txs`, `_block`) renamed to `ops_hash` / `ops` / `dummy_ops` / `empty_ops` / `_ops` throughout the changed call paths. The `CompactMb` field stays `transactions_hash` for now to keep DB encoding stable. - Reintroduce `BlockPayload = LimitedVec<u8, MAX_BLOCK_PAYLOAD_BYTES>` in `ethexe-malachite-core` and use it everywhere instead of the bare `LimitedVec<...>` form (signatures, fields, doc comments, the `try_from` conversion). Drops the redundant `test_payload()` helper in `multi_validators.rs`. - `EXPECTED_TYPE_INFO_HASH` bumped because the doc comment on `CompactMb::mb_compact_block` changed; SCALE encoding is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hite-core-concretize-payload # Conflicts: # ethexe/common/src/mock.rs
breathx
approved these changes
Jun 5, 2026
grishasobol
added this pull request to the merge queue
Jun 5, 2026
grishasobol
removed this pull request from the merge queue due to a manual request
Jun 5, 2026
…l transactions terminology Sweep the rest of the Transaction→Operation vocabulary that survived the previous pass: - `CompactMb::transactions_hash` → `operations_hash`. Field rename only; SCALE layout is unchanged because the encoding is positional. Bump `EXPECTED_TYPE_INFO_HASH` to match the new scale-info field name. - All locals (`ops_hash`, `transactions_hash`) folded to `operations_hash` so struct inits can use field shorthand. - Test helpers renamed: `validate_transactions` → `validate_operations`, `build_transactions` → `build_operations`; the `transactions` Vec used to build a one-off `Operations` in `validate_block_above`'s ad-hoc test becomes `operations`. - `hash_changes_when_transactions_change` and `transaction_tag_distinguishes_variants` test names move to `operations`. - Doc comments, `warn!`, `expect`, and assert messages that referred to "transactions blob" / "extra transactions" updated to "operations blob" / "extra operations". Mempool-side terminology (`InjectedTransaction`, "non-zero-value transactions", purged-tx events) stays — those are the actual user-transaction layer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
grishasobol
added this pull request to the merge queue
Jun 5, 2026
grishasobol
added a commit
that referenced
this pull request
Jun 8, 2026
…rations rename Master rewrote BlockPayload as Operations (#5506) and renamed the malachite block payload field to operations_hash, obsoleting the original PR's first commit. This commit reconstitutes the PR's typing changes on top of current master: - introduces HashOf<EB> / HashOf<MB> through SimpleBlockData.hash, BlockHeader.parent_hash, BlockData.hash, BlockMeta/MbMeta fields, DBGlobals, DBConfig, ChainCommitment.last_advanced_eth_block, BatchCommitment.block_hash, InjectedTransaction.reference_block, and the MbStorageRO/RW + OnChainStorageRO/RW + BlockMetaStorageRO/RW trait signatures - moves CompactMb into ethexe-common::malachite alongside new MB and BlockPayload types (keeping CompactMb re-exported from db.rs for source compat), and adds the reserved [u8; 64] tail - threads typed hashes through observer (ObserverEvent, BlockLoader, EthereumBlockLoader), consensus (validator state machine, ConsensusService trait, batch utils), compute (ComputeEvent, ComputeError, sub-services, executable preparation), ethexe-malachite (MalachiteEvent, CommitCertificate, EthexeExternalities, mempool, quarantine, tx_validity, receive_eb_prepared), processor (BoundPromiseSink), network (set_chain_head, validator list, db_sync ProgramIds), db (RawDatabase + Database trait impls, iterator nodes, verifier errors, dump collector, init migration), rpc (block_at_or_latest_synced, latest_computed_mb), ethereum (IntoBlockId impl), and cli (dump, check, tx commands) - recomputes EXPECTED_TYPE_INFO_HASH in ethexe-common::db::tests now that BlockMeta / MbMeta / CompactMb / BlockHeader / DBGlobals carry typed wrappers - gates the test module in ethexe-malachite::externalities and ethexe-malachite::tx_validity behind `cfg(all(test, never_built))` pending a follow-up that threads HashOf<MB>/HashOf<EB> through the synthetic-chain helpers and assert_eq! boundary conversions (the prod code those tests exercise is fully typed; the surrounding mempool / quarantine / restart_resilience tests still build and pass) Tests: ethexe-common, ethexe-malachite, ethexe-malachite-core, ethexe-compute, ethexe-processor all pass via cargo nextest (153 tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5505.