feat: user batch support by Mirko-von-Leipzig · Pull Request #1846 · 0xMiden/node

Mirko-von-Leipzig · 2026-03-26T16:25:14Z

This PR is the third and final part of the mempool refactoring PR stack. Part 1 (#1820) performs the broad mempool refactoring to simplify this PR. Builds on part 2 (#1832).

Batch submissions must include their transaction inputs since we currently require this for the validator to verify them before inclusion in a block. This PR abuses this by treating the batch as a set of normal transactions at the mempool level. This simplifies the mempool implementation, which is currently built around a DAG of transactions - so having to insert a batch directly would be more complex. This will need to change once we stop requiring transaction inputs as part of the validator; but it won't be too bad.

The way this is implemented here, is that the transaction DAG tracks user batches and ensures that when a batch is selected, that transactions from user batches are not mixed with conventional transactions. That is, select_batch outputs either a user batch, or a conventional batch.

Effectively, the transaction DAG internally ensures that the user batch's transactions remain coherent even though the batch has been deconstructed into individual transactions. The benefit is that this doesn't require any major structural changes to the mempool. The rest of the mempool then treats the user batch as per normal.

Closes #1112

Mirko-von-Leipzig · 2026-03-26T16:32:33Z

proto/proto/types/transaction.proto

-    // Encoded using [winter_utils::Serializable] implementation for
-    // [miden_protocol::transaction::proven_tx::ProvenTransaction].
-    bytes encoded = 1;
+message TransactionBatch {


The suggestion here I think was to re-add the proven_batch property, and have the others be optional so we can drop them at some point.

This is probably fine for now, but in the next release, we'll need to change this to something like ProvenTransactionBatch which would contain the ProvenBatch struct + optional data that would allow us to re-execute all transactions in the batch and validate that the proven batch is correct (similar to how we do it for proven transactions).

Mirko-von-Leipzig · 2026-03-26T16:36:08Z

crates/rpc/src/server/api.rs

@PhilippGackstatter if you could throw an eye on the process here to ensure I'm checking the correct things.

The state itself is checked in the mempool, so here we really just want to ensure that the batch and its transactions are valid and the reference block is correct iiuc.

Mirko-von-Leipzig · 2026-03-26T16:36:26Z

crates/rpc/src/server/api.rs

+        let reference_commitment: Word = reference_header
+            .chain_commitment
+            .expect("store should always fill block header")
+            .try_into()
+            .expect("store Word should be okay");
+        if reference_commitment != proof.reference_block_commitment() {


Is this correct?

I don't think it is, here chain_commitment is the root of the blockchain MMR, whereas the reference_block_commitment sounds more like it's the block hash.

crates/block-producer/src/mempool/graph/transaction.rs

sergerad · 2026-03-27T01:10:06Z

crates/block-producer/src/mempool/mod.rs

+        &mut self,
+        txs: &[Arc<AuthenticatedTransaction>],
+    ) -> Result<BlockNumber, MempoolSubmissionError> {
+        assert!(!txs.is_empty(), "Cannot have a batch with no transactions");


Just checking we want to crash here instead of return error

I assumed that one cannot build a ProvenBatch without one, so this would indicate an internal bug somewhere. But maybe that's a poor assumption.

sergerad · 2026-03-27T01:11:38Z

crates/block-producer/src/mempool/graph/transaction.rs

+    }
+
+    pub fn select_batch(&mut self, budget: BatchBudget) -> Option<SelectedBatch> {
+        self.select_user_batch().or_else(|| self.select_conventional_batch(budget))


Might want some doc comments to make it clear that budget is intended to only relevant for conventional batches.

Also are we OK with user batches always taking priority over conventional here?

Also wondering if we need to prevent user batches of size 1 (or some other limit). Unsure if that is relevant to this PR just a general thought.

Also are we OK with user batches always taking priority over conventional here?

I'm unsure, but at the moment it doesn't matter much. If its a concern we can make it random -- I was thinking maybe that's best.

Also wondering if we need to prevent user batches of size 1 (or some other limit). Unsure if that is relevant to this PR just a general thought.

Good question. I'm unsure 😬 I wonder if that makes some user loop more difficult i.e. they always submit user batches, but sometimes they don't have many transactions to bundle..

Probably we would want some limit even in the future? cc @bobbinth

Also are we OK with user batches always taking priority over conventional here?

Would having fees solve this?

crates/block-producer/src/mempool/mod.rs

crates/block-producer/src/server/mod.rs

igamigo

LGTM! I left some mostly minor, non-blocking comments. Not sure if you tested already but jsut in case, I'm going to run the client integration tests and report back.

igamigo · 2026-03-27T18:16:00Z

crates/block-producer/src/mempool/graph/batch.rs

    /// graph.
-    pub fn prune(&mut self, batch: BatchId) {
-        self.inner.prune(batch);
+    pub fn prune(&mut self, batch: BatchId) -> SelectedBatch {


nit: Would be nice to update the doc comments to say that this returns the pruned batch now

igamigo · 2026-03-27T18:16:33Z

crates/block-producer/src/mempool/graph/dag.rs

    /// # Panics
    ///
    /// Panics if this node has any ancestor nodes, or if this node was not selected.
-    pub fn prune(&mut self, id: N::Id) {


igamigo · 2026-03-27T18:30:42Z

crates/block-producer/src/mempool/graph/transaction.rs

+    }
+
+    pub fn select_batch(&mut self, budget: BatchBudget) -> Option<SelectedBatch> {
+        self.select_user_batch().or_else(|| self.select_conventional_batch(budget))


Also are we OK with user batches always taking priority over conventional here?

Would having fees solve this?

igamigo · 2026-03-27T18:41:56Z

crates/block-producer/src/mempool/graph/transaction.rs

+                .expect("bi-directional mapping should be coherent");
+
+            for tx in txs {
+                let Some(tx) = self.inner.selection_candidates().get(&tx).copied() else {


Maybe this calls for a get_selection_candidate()? So that you avoid re-allocating the map every iteration through selection_candidates()

igamigo · 2026-03-27T18:46:39Z

crates/block-producer/src/mempool/graph/transaction.rs

+            // Select arbitrary candidate which is _not_ part of a user batch.
+            let candidates = self.inner.selection_candidates();


This is not important now and I know this was pre-existing but re-allocating on every iteration here as well is probably not ideal

igamigo · 2026-03-27T19:05:01Z

crates/block-producer/src/server/mod.rs

+        // We assume that the rpc component has verified everything, including the transaction
+        // proofs.


nit, feel free to disregard: this is a bit redundant considering the safety comment near the end of the function

igamigo · 2026-03-27T19:19:51Z

crates/rpc/src/server/api.rs

+            .await?
+            .into_inner()
+            .block_header
+            .expect("store should always send block header");


I don't think this is true for blocks beyond the chain tip, so should this be an error?

igamigo · 2026-03-27T19:25:17Z

crates/rpc/src/server/api.rs

+        let reference_commitment: Word = reference_header
+            .chain_commitment
+            .expect("store should always fill block header")
+            .try_into()
+            .expect("store Word should be okay");
+        if reference_commitment != proof.reference_block_commitment() {


I don't think it is, here chain_commitment is the root of the blockchain MMR, whereas the reference_block_commitment sounds more like it's the block hash.

Mirko-von-Leipzig added 7 commits March 26, 2026 14:40

User batch gRPC now includes txs for validation

5e84013

Thread through to mempool

6691da9

Basic appending to transaction graph

a1f2066

Implement user batches within TransactionGraph

9786578

Check batch budget

ba58888

Improve mempool error naming

d656f18

Add tests

4b264df

Mirko-von-Leipzig force-pushed the mirko/mempool-user-batches branch from bf86aec to 51e74e4 Compare March 26, 2026 16:29

changelog

6dd5f53

Mirko-von-Leipzig force-pushed the mirko/mempool-user-batches branch from 51e74e4 to 6dd5f53 Compare March 26, 2026 16:31

Mirko-von-Leipzig commented Mar 26, 2026

View reviewed changes

Mirko-von-Leipzig marked this pull request as ready for review March 26, 2026 16:34

Mirko-von-Leipzig requested review from PhilippGackstatter, bobbinth, igamigo and sergerad March 26, 2026 16:34

Mirko-von-Leipzig commented Mar 26, 2026

View reviewed changes

sergerad reviewed Mar 27, 2026

View reviewed changes

crates/block-producer/src/mempool/graph/transaction.rs Outdated Show resolved Hide resolved

sergerad reviewed Mar 27, 2026

View reviewed changes

crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved

sergerad reviewed Mar 27, 2026

View reviewed changes

crates/block-producer/src/server/mod.rs Show resolved Hide resolved

Serge review

681eb87

igamigo approved these changes Mar 27, 2026

View reviewed changes

		// Select arbitrary candidate which is _not_ part of a user batch.
		let candidates = self.inner.selection_candidates();

		// We assume that the rpc component has verified everything, including the transaction
		// proofs.

Conversation

Mirko-von-Leipzig commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

igamigo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Mirko-von-Leipzig commented Mar 26, 2026 •

edited

Loading