Skip to content

gossip: refactor peer sampling and fix invalid active set state #6753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

gossip: refactor peer sampling and fix invalid active set state #6753

wants to merge 3 commits into from

Conversation

@ravyu-jump
Copy link
Contributor

@ravyu-jump ravyu-jump commented Oct 14, 2025 •
edited
Loading

  • Convert wpeer sampler into a standalone library: used by CRDS peer sampler and Active Set bucket samplers
  • Add callback fd_crds_ci_change_fn that tracks individual contact info events (insert/drop, stake change, identity change)
  • Refactor active set: Active set is fully contained, maintaining samplers, push state, and state machines (with APIs to update internal state).
  • Changes collectively fix invalid active set states where:
    • a dropped peer might still be in a bucket
    • if node's identity change is triggered, the new identity might still be in a sample set or bucket

@github-actions
Copy link

github-actions bot commented Oct 14, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.079726 s 0.08035 s 0.783% ✅
backtest mainnet-368528500-perf snapshot load 2.887 s 2.865 s -0.762% ✅
backtest mainnet-368528500-perf total elapsed 79.726319 s 80.349917 s 0.782% ✅
firedancer mem usage with mainnet.toml 1017.12 GiB 1017.12 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from 2cddf35 to 9738713 Compare October 15, 2025 00:00
@github-actions
Copy link

github-actions bot commented Oct 15, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.079669 s 0.079646 s -0.029% ✅
backtest mainnet-368528500-perf snapshot load 2.905 s 2.894 s -0.379% ✅
backtest mainnet-368528500-perf total elapsed 79.668874 s 79.645919 s -0.029% ✅
firedancer mem usage with mainnet.toml 1017.12 GiB 1017.12 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from 9738713 to e020e42 Compare October 15, 2025 16:06
@github-actions
Copy link

github-actions bot commented Oct 15, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.079524 s 0.079394 s -0.163% ✅
backtest mainnet-368528500-perf snapshot load 2.892 s 2.901 s 0.311% ✅
backtest mainnet-368528500-perf total elapsed 79.524383 s 79.393759 s -0.164% ✅
firedancer mem usage with mainnet.toml 1017.12 GiB 1017.12 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from e020e42 to 4f31206 Compare October 15, 2025 20:54
@github-actions
Copy link

github-actions bot commented Oct 15, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.07899 s 0.07867 s -0.405% ✅
backtest mainnet-368528500-perf snapshot load 2.925 s 2.875 s -1.709% ✅
backtest mainnet-368528500-perf total elapsed 78.989667 s 78.669711 s -0.405% ✅
firedancer mem usage with mainnet.toml 1017.12 GiB 1017.12 GiB 0.000% ✅

mmcgee-jump
mmcgee-jump requested changes Oct 15, 2025
View reviewed changes
Copy link
Contributor

@mmcgee-jump mmcgee-jump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handle identity change

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from 4f31206 to c882a67 Compare October 16, 2025 20:41
@github-actions
Copy link

github-actions bot commented Oct 16, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.066881 s 0.066521 s -0.538% ✅
backtest mainnet-368528500-perf snapshot load 2.239 s 2.211 s -1.251% ✅
backtest mainnet-368528500-perf total elapsed 66.880996 s 66.520509 s -0.539% ✅
firedancer mem usage with mainnet.toml 1021.13 GiB 1019.13 GiB -0.196% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from c882a67 to adf7f8d Compare October 22, 2025 17:32
@github-actions
Copy link

github-actions bot commented Oct 22, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.063497 s 0.050606 s -20.302% ✅
backtest mainnet-368528500-perf snapshot load 1.991 s 1.548 s -22.250% ✅
backtest mainnet-368528500-perf total elapsed 63.496859 s 50.606313 s -20.301% ✅
firedancer mem usage with mainnet.toml 1013.13 GiB 1013.13 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from adf7f8d to 7a41482 Compare October 22, 2025 19:38
@github-actions
Copy link

github-actions bot commented Oct 22, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.053015 s 0.052585 s -0.811% ✅
backtest mainnet-368528500-perf snapshot load 1.758 s 1.734 s -1.365% ✅
backtest mainnet-368528500-perf total elapsed 53.014664 s 52.585069 s -0.810% ✅
firedancer mem usage with mainnet.toml 1013.13 GiB 1013.13 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch 2 times, most recently from a59f038 to 7ced4a9 Compare October 22, 2025 22:20
@github-actions
Copy link

github-actions bot commented Oct 22, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.052719 s 0.053274 s 1.053% ✅
backtest mainnet-368528500-perf snapshot load 1.718 s 1.735 s 0.990% ✅
backtest mainnet-368528500-perf total elapsed 52.71897 s 53.274178 s 1.053% ✅
firedancer mem usage with mainnet.toml 1013.13 GiB 1013.13 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from 7ced4a9 to fd8a06a Compare October 23, 2025 19:29
@github-actions
Copy link

github-actions bot commented Oct 23, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.052746 s 0.052558 s -0.356% ✅
backtest mainnet-368528500-perf snapshot load 1.733 s 1.772 s 2.250% ✅
backtest mainnet-368528500-perf total elapsed 52.745641 s 52.558139 s -0.355% ✅
firedancer mem usage with mainnet.toml 1014.1 GiB 1014.1 GiB 0.000% ✅

@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from fd8a06a to 702d554 Compare November 12, 2025 22:52
@github-actions
Copy link

github-actions bot commented Nov 12, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.054327 s 0.054555 s 0.420% ✅
backtest mainnet-368528500-perf snapshot load 2.608 s 1.711 s -34.394% ✅
backtest mainnet-368528500-perf total elapsed 54.327382 s 54.555083 s 0.419% ✅
firedancer mem usage with mainnet.toml 996.14 GiB 996.14 GiB 0.000% ✅

@ravyu-jump ravyu-jump linked an issue Nov 20, 2025 that may be closed by this pull request
gossip: refactor sampler management #7345
Open
@ravyu-jump ravyu-jump force-pushed the gossip/sampler-fix-state-machine branch from 702d554 to 5a31b0c Compare November 20, 2025 19:11
@github-actions
Copy link

github-actions bot commented Nov 20, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.053256 s 0.053175 s -0.152% ✅
backtest mainnet-368528500-perf snapshot load 2.601 s 1.815 s -30.219% ✅
backtest mainnet-368528500-perf total elapsed 53.255594 s 53.174616 s -0.152% ✅
firedancer mem usage with mainnet.toml 1001.1 GiB 1001.1 GiB 0.000% ✅

@ravyu-jump ravyu-jump mentioned this pull request Nov 24, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mmcgee-jump mmcgee-jump mmcgee-jump requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

gossip: refactor sampler management

3 participants

@ravyu-jump @mmcgee-jump