log :: 2025‐12

Tip

KEYWORDS

Mini-Protocols, Pure-Stage, Tx Submission, ChainSync, NetworkResource / Mempool

SUMMARY

Handshake, keepalive, txsubmission, and chainsync are implemented as stages (mostly pure-stage) with protocol state in amaru-network and event processing in amaru-consensus; txsubmission hooks into a TxSubmissionMempool via a centralized NetworkResource.

The network stack now avoids pallas-network for handshake/keepalive (precise wire types), while txsubmission depends on a pending Pallas PR; handshake compat with the Haskell node is a known blocker.

Client/server txsubmission wiring is progressing (effects, actors, simulation mocks), with plans to test via a mempool that replays transactions from stored blocks.

Operational notes: large PRs scheduled for group review; build tuned (codegen-units=8) to reduce compile time, to be revisited for runtime performance.

2025-12-19

Mini-protocols

• The handshake stage is implemented as a pure-stage stage in #603.

  • A connection stage manages the connection, starts the Muxer for that connection, and initializes the handshake mini-protocol.

• The keepalive protocol is implemented as a pure-stage stage in #608.

• The txsubmission protocol is implemented as a stage in #611

  • For now the sizes and timeouts limits are not enforced.
  • Additionally a pass over the existing Haskell node must be done to check if some important implementation decisions not present in the network specification should be ported over to the amaru node.

• The chainsync protocol is implemented as a stage in #612.

• Notes:

  • The connection, handshake and keepalive protocols are implemented in the amaru-network crate and don't rely on the pallas-network crate anymore. This allows us to precisely control the data types used on the wire and internally.

  • The txsubmission protocol relies on a network resource (in pure-stage terms) that offers a TxSubmissionMempool interface to the Mempool.

    • The Mempool itself is implemented in the amaru-mempool crate.
    • Everything is eventually wired-up in the amaru-consensus crate.

  • The chainsync protocol require more stages for its full processing, with stages to validate headers, fetch blocks, etc... The code is divided in two parts:

    • The management of the protocol state is done in the amaru-network crate. That stage emits events to a stage reference that does the rest of the processing.
    • The processing of chain sync events is then done with several stages in the amaru-consensus crate.

  • The overall dependencies are:

  +-----------+
  | consensus |-----+-------------+
  +-----------+     |             |
      |             v             v
      |         +---------+    +---------+
      |         | network |    | mempool |
      |         +---------+    +---------+
      |             |             |
      v             v             |
  +------------------+            |
  | ouroboros-traits |<-----------+
  +------------------+

2025-12-10

Tx submission protocol

• The PR implementing a first pass at the protocol is ready for review: https://github.com/pragma-org/amaru/pull/587.
  • The PR description details the main modifications and has a few implementation notes.
• This is a pretty large PR so we will do a group review with Roland and Arnaud on Friday.
• One blocker will be the merge of this Pallas PR: https://github.com/txpipe/pallas/pull/715, since the current code depends on it.
• The next important step will be to fix the handshake protocol between the Amaru node and the Haskell node since it prevents us from testing the tx submission protocol.

2025-12-05

Tx submission protocol -> server integration with stages

• I did a bit of refactor to move all the network effects to one place NetworkResource with an UpstreamClient containing actors and clients to connect to upstream peers + DownstreamServer to serve downstream peers.
• Then I had an issue building amaru with the release profile. The build takes forever. I eventually set codegen-units = 8 to bring the compilation time down to 2m40s. This will have to be fixed though since changing this parameter has an impact on runtime performance.
• When I tried to connect nodes to do some testing I realized something: there are no transactions in the mempool at the moment :-).
  • They could come from the local tx submission protocol but this will be implemented later.
  • I guess the best way to test for now will be to have a Mempool implementation that grabs a block from store, and returns transactions from that block, in order to send them to an Amaru node upstream. Since the Mempool does not yet validate transactions that should be fine.
  • That being said, I'm observing a disconnection from the upstream Haskell node at the mempool and I don't see a reason why that should happen The Haskell node should just wait until transactions are available.

2025-12-04

Tx submission protocol -> server integration with stages

• I took the same approach as the one taken for the client:
  • Create a pull stage looping with a NextTxRequest message.
  • On each loop a ReceiveTxReplyEffect is triggered to get the next TxReply from a downstream client.
  • The effect is implemented with the NetworkResource and eventually with the actor code accepting client requests.
• I removed the ForwardListenerResource that was used to implement a ForwardEventEffect (to send chain sync events downstream).
  • This functionality is now supported by the NetworkResource in order to keep all network effects close.
• The final connection between the NetworkResource and the actor code is not done and there are also changes to some mocks in the simulation code (those mocks avoid network calls).

2025-12-03

Tx submission protocol -> client integration with stages

• I had to take a detour to add some useful instances to some pallas crates:
  • One PR to add instances on top of main. But we cannot use it for now, it has too many changes.
  • One PR to add instances on top of lts/v0 (those are slightly different).
  • This means that my current PR relies on a branch at the moment but Santiago told me during office hours that he should merge those PRs quite quickly
    • Note: he said that releasing the lts version (as 0.34.0) might not happen right away but he would do it if we needed to.
• I have integrated the client side of the protocol as a stage with the necessary "mempool" and "network" effects to support the simulation.
• Next: the integration of the server side of the protocol

2025-12-02

Tx submission protocol -> client integration with stages

• For the integration of the tx submission client, I chose to use the same strategy as what we currently do for getting headers from upstream:
  • A pull_tx_request stage loops with a single NextTxRequest message.
  • On each loop it performs a TxRequestEffect implemented by the existing NetworkOperations trait with an additional method: next_tx_request.
  • NetworkOperations is implemented as a NetworkResource. That data type makes connections to upstream peers and maintains a set of mini-protocol clients per peer.
  • I've extended that code to have a Receiver<TxRequest> that receives a new request every time the underlying Pallas client gets a Request<EraTxId, EraTxBody>.
• Then the stage implementation consists in reusing the code I already have in the TxSubmissionClient by:
  • Isolating the "mempool" effects (wait_until_tx_ready, get_tx_ids etc...)
  • Processing each TxRequest by interacting with the Mempool
  • Using additional network effects to send back responses => This part is not yet wired.
  • Note that the stage contains a Map<Peer, TxSubmissionClient>. Eventually it will also contain some logic to control the behavior we require across all peers.
  • Concretely I've been a bit stuck trying to work-around some missing instances in pallas. My initial attempts were ok but the requirements we have in terms of serialization for the simulation framework make that really cumbersome. After reaching out to Santiago @ TxPipe we agreed to merge the changes in this PR: https://github.com/txpipe/pallas/pull/714 to the long-term maintenance branch for pallas-0.x. This should unblock me and simplify a few things.
• Next steps:
  • Get everything to compile with the LTS Pallas branch.
  • Complete the execution path for replies
  • Integrate the server