bundle: defer publish to after_credit via pending deque by cmoyes-jump · Pull Request #8763 · firedancer-io/firedancer

cmoyes-jump · 2026-03-06T19:22:09Z

Split into before_credit (I/O, gated on deque empty and cr_avail) and after_credit (drain one bundle or up to STEM_BURST packets per call). Publish metadata is buffered in a zero-copy deque. dcache writes are capped at cr_avail to prevent overwriting data the downstream consumer hasn't read.

Fixes #8654

Copilot

Pull request overview

This PR refactors the Firedancer bundle tile's publish path to separate gRPC I/O (before_credit) from stem publishing (after_credit). Instead of publishing directly to the tango message bus during gRPC callbacks, transactions are written to the dcache and their metadata is buffered in a zero-copy dynamic deque (pending_pubs). The after_credit callback then drains the deque: publishing complete bundles atomically (up to STEM_BURST) or batches of up to STEM_BURST individual packets per call. Two new metrics expose the pending buffer depth and any backpressure-induced drops.

Changes:

New fd_bundle_pending_pub deque in the bundle tile's private state, used to stage publish metadata between gRPC I/O and fd_stem_publish.
New before_credit/after_credit split: I/O is gated on the deque being empty and at least 1 credit being available; draining is gated on min_cr_avail >= STEM_BURST.
New metrics (PendingTransactions gauge, TransactionDroppedBackpressure counter) and associated generated code and documentation.

Reviewed changes

Copilot reviewed 7 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/disco/bundle/fd_bundle_tile.c`	Adds `before_credit`, moves `STEM_BURST` earlier, splits `after_credit` into drain loop + plugin update
`src/disco/bundle/fd_bundle_tile_private.h`	Adds `fd_bundle_pending_pub` struct, deque template, `depth` field in `fd_bundle_out_ctx`, `backpressure_drop_cnt` metric
`src/disco/bundle/fd_bundle_client.c`	Replaces direct `fd_stem_publish` calls with deque pushes; adds per-publish backpressure checks
`src/disco/bundle/test_bundle_common.c`	Allocates/destroys the deque in the test harness, adds `depth` to test verify_out
`src/disco/bundle/test_bundle_client.c`	Updates existing tests to check deque count instead of mcache; adds three new drain-logic unit tests
`src/disco/metrics/metrics.xml`	Adds two new bundle metrics and two unrelated snapct gossip metrics
`src/disco/metrics/generated/fd_metrics_bundle.h`	Regenerated header reflecting new metric offsets
`src/disco/metrics/generated/fd_metrics_bundle.c`	Regenerated source with new `DECLARE_METRIC` entries
`book/api/metrics-generated.md`	Documentation updated for new bundle and snapct metrics

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

Copilot reviewed 7 out of 11 changed files in this pull request and generated 4 comments.

You can also share your feedback on Copilot code review. Take the survey.

jherrera-jump

Thanks for looking into this!

jherrera-jump · 2026-03-06T19:44:17Z

    <histogram name="MessageRxDelayNanos" min="100000" max="1000000000">
        <summary>Message receive delay in nanoseconds from bundle server to bundle client</summary>
    </histogram>
+    <gauge name="PendingTransactions" summary="Number of transactions buffered and waiting to be published" />


PendingTransactions here is probably less useful as a metric due to how slow consumers poll metrics compared to the rate of change of the metric itself.

Maybe instead this should be the number of times pending_transaction_cnt exceeds pending_max/2?

Also don't feel too strongly about this so fine to leave it as-is.

if others also feel this is worth doing, I will change, how does that sound?

Copilot

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated 2 comments.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-06T20:48:00Z

+  ulong pending_max = tile->bundle.buf_sz / FD_BUNDLE_MIN_GRPC_WIRE_SZ;
  ulong l = FD_LAYOUT_INIT;
  l = FD_LAYOUT_APPEND( l, alignof(fd_bundle_tile_t), sizeof(fd_bundle_tile_t)                        );
  l = FD_LAYOUT_APPEND( l, fd_grpc_client_align(),    fd_grpc_client_footprint( tile->bundle.buf_sz ) );
  l = FD_LAYOUT_APPEND( l, fd_alloc_align(),          fd_alloc_footprint()                            );
+  l = FD_LAYOUT_APPEND( l, pending_txn_align(),       pending_txn_footprint( pending_max )            );


The pending transaction deque is sized as buf_sz / FD_BUNDLE_MIN_GRPC_WIRE_SZ entries, but each fd_bundle_pending_txn_t entry occupies approximately 1304 bytes (1232 bytes payload + metadata). For the default buffer_size_kib = 2048 (fdctl), this yields ~131,072 entries × 1304 bytes ≈ 171 MB just for the deque. For buffer_size_kib = 16384 (firedancer), it yields ~1.37 GB.

The sizing rationale is correct for overflow prevention (the gRPC frame buffer limits inbound transactions), but the deque entries are ~81x larger than the minimum wire format assumption (16 bytes). In practice, a much smaller deque (e.g., capped at a few hundred entries, since at most STEM_BURST * N transactions are needed to keep the pipeline full) would suffice and avoid this large memory footprint in the tile's scratch space.

need guidance on whether we can safely size down the theoretical upper bound

I think you can just size to match the bundle_verif link, i.e. config->tiles.verify.receive_buffer_size / FD_TPU_PARSED_MTU

I agree, that makes sense, I made your suggested change

Copilot

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated 1 comment.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

Copilot reviewed 11 out of 13 changed files in this pull request and generated 1 comment.

You can also share your feedback on Copilot code review. Take the survey.

unit tests zero copy design more hardening

Co-authored-by: ripatel-fd <ripatel+git@jumptrading.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 13 changed files in this pull request and generated 2 comments.

You can also share your feedback on Copilot code review. Take the survey.

cmoyes-jump self-assigned this Mar 6, 2026

Copilot AI review requested due to automatic review settings March 6, 2026 19:22

Copilot started reviewing on behalf of cmoyes-jump March 6, 2026 19:22 View session

cmoyes-jump force-pushed the cmoyes/bundletile branch 3 times, most recently from 4a7245c to 25d8077 Compare March 6, 2026 19:26

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_tile.c Outdated

Comment thread src/disco/bundle/fd_bundle_tile.c Outdated

Copilot AI review requested due to automatic review settings March 6, 2026 19:30

Copilot started reviewing on behalf of cmoyes-jump March 6, 2026 19:30 View session

cmoyes-jump force-pushed the cmoyes/bundletile branch from 3487482 to 0445c02 Compare March 6, 2026 19:33

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_tile.c Outdated

Comment thread src/disco/bundle/fd_bundle_tile_private.h

Comment thread src/disco/bundle/test_bundle_client.c

Comment thread src/disco/metrics/metrics.xml Outdated

cmoyes-jump requested review from mmcgee-jump and ripatel-fd March 6, 2026 19:50

mmcgee-jump reviewed Mar 6, 2026

View reviewed changes

Comment thread src/disco/metrics/metrics.xml Outdated

jherrera-jump requested changes Mar 6, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 6, 2026 20:12

Copilot started reviewing on behalf of cmoyes-jump March 6, 2026 20:13 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 6, 2026 20:40

cmoyes-jump force-pushed the cmoyes/bundletile branch from 6d8ae29 to 3a9cf13 Compare March 6, 2026 20:40

Copilot started reviewing on behalf of cmoyes-jump March 6, 2026 20:40 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 6, 2026 20:51

Copilot started reviewing on behalf of cmoyes-jump March 6, 2026 20:51 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_tile_private.h

cmoyes-jump force-pushed the cmoyes/bundletile branch from fea47fe to c728d5f Compare March 7, 2026 19:37

Copilot AI review requested due to automatic review settings March 7, 2026 19:45

Copilot started reviewing on behalf of cmoyes-jump March 7, 2026 19:45 View session

Copilot AI reviewed Mar 7, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_client.c

ripatel-fd reviewed Mar 8, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_tile.c Outdated

ripatel-fd previously approved these changes Mar 8, 2026

View reviewed changes

cmoyes-jump force-pushed the cmoyes/bundletile branch from ea82ac1 to 790db69 Compare March 8, 2026 23:20

cmoyes-jump added 9 commits March 9, 2026 09:33

bundle: fix STEM_BURST by buffering transactions in a deque

493893d

unit tests zero copy design more hardening

tile

1156ad0

PR comments

365e386

use stem abstraction

ed2e0db

PR comments

7aa8092

remove stale metrics

d5f5abe

remove stack copies

9308461

add tests

cc3c08c

plumb bundle topology depth

41dedb4

Copilot AI review requested due to automatic review settings March 9, 2026 14:33

cmoyes-jump force-pushed the cmoyes/bundletile branch from 790db69 to 41dedb4 Compare March 9, 2026 14:33

cmoyes-jump enabled auto-merge (squash) March 9, 2026 14:33

Copilot started reviewing on behalf of cmoyes-jump March 9, 2026 14:33 View session

Update src/disco/bundle/fd_bundle_tile.c

7030b9e

Co-authored-by: ripatel-fd <ripatel+git@jumptrading.com>

cmoyes-jump dismissed ripatel-fd’s stale review via 7030b9e March 9, 2026 14:34

Copilot AI reviewed Mar 9, 2026

View reviewed changes

Comment thread src/disco/bundle/fd_bundle_client.c Outdated

Comment thread src/disco/bundle/fd_bundle_tile.c

cmoyes-jump added 2 commits March 9, 2026 14:49

late breaking nits

7bdf900

PR late breaking

e43daa3

jherrera-jump approved these changes Mar 9, 2026

View reviewed changes

cmoyes-jump merged commit b7f5790 into firedancer-io:main Mar 9, 2026
18 checks passed

Conversation

cmoyes-jump commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jherrera-jump left a comment

Choose a reason for hiding this comment

Uh oh!

jherrera-jump Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

cmoyes-jump Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

cmoyes-jump Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

jherrera-jump Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

cmoyes-jump Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

cmoyes-jump commented Mar 6, 2026 •

edited

Loading

cmoyes-jump Mar 6, 2026 •

edited

Loading

cmoyes-jump Mar 7, 2026 •

edited

Loading