Skip to content

feat(server-ng): add cluster.heartbeat_timeout for VSR liveness window - #3738

Merged
hubcio merged 2 commits into
masterfrom
feat/surface-heartbeat-timeout
Jul 23, 2026
Merged

feat(server-ng): add cluster.heartbeat_timeout for VSR liveness window#3738
hubcio merged 2 commits into
masterfrom
feat/surface-heartbeat-timeout

Conversation

@hubcio

@hubcio hubcio commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

The backup-side primary-liveness window was hardcoded at 5s
(NORMAL_HEARTBEAT_TICKS). On oversubscribed hosts scheduling stalls
exceed it, so backups elect a new primary although the old one is
healthy, and client sessions die with the view change - loaded
parallel integration runs failed 23 message_retrieval tests this
way.

Surface the window as [cluster] heartbeat_timeout on the server-ng
config. ClusterConfig is forked into server_ng_config (like the
tcp/quic/websocket forks) because the legacy struct sources its
defaults from the legacy config.toml, which is frozen for ng-only
knobs. The runtime converts the duration to consensus ticks and
installs it before init, a validator enforces a 2s floor (the zero
sentinels are rejected too), and a unit test pins the TOML default
to the consensus built-in. The retrieval scenarios now run with a
60s window so box load cannot fake primary death.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jul 23, 2026
numinnex
numinnex previously approved these changes Jul 23, 2026
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.93333% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.40%. Comparing base (2fdd6e8) to head (32727fa).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
core/configs/src/server_ng_config/cluster.rs 96.72% 9 Missing and 1 partial ⚠️
core/server-ng/src/bootstrap.rs 59.09% 9 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             master    #3738       +/-   ##
=============================================
- Coverage     74.63%   60.40%   -14.23%     
  Complexity      950      950               
=============================================
  Files          1303     1303               
  Lines        149183   137150    -12033     
  Branches     124694   112735    -11959     
=============================================
- Hits         111338    82845    -28493     
- Misses        34348    50720    +16372     
- Partials       3497     3585       +88     
Components Coverage Δ
Rust Core 57.51% <94.93%> (-17.41%) ⬇️
Java SDK 62.64% <ø> (ø)
C# SDK 71.13% <ø> (-1.16%) ⬇️
Python SDK 92.27% <ø> (ø)
PHP SDK 84.52% <ø> (ø)
Node SDK 92.70% <ø> (ø)
Go SDK 43.08% <ø> (ø)
Files with missing lines Coverage Δ
core/configs/src/server_ng_config/defaults.rs 100.00% <100.00%> (ø)
core/configs/src/server_ng_config/server_ng.rs 47.05% <ø> (ø)
core/consensus/src/impls.rs 75.11% <100.00%> (+0.04%) ⬆️
core/consensus/src/lib.rs 0.00% <ø> (ø)
core/consensus/src/vsr_timeout.rs 94.67% <100.00%> (+0.09%) ⬆️
core/server-ng/src/cluster_meta.rs 15.87% <ø> (ø)
core/server-ng/src/http.rs 51.89% <ø> (ø)
core/server-ng/src/http/error.rs 31.86% <ø> (ø)
core/server-ng/src/partition_helpers.rs 64.74% <100.00%> (+0.10%) ⬆️
core/server-ng/src/bootstrap.rs 11.71% <59.09%> (+0.56%) ⬆️
... and 1 more

... and 259 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mmodzelewski
mmodzelewski previously approved these changes Jul 23, 2026
The backup-side primary-liveness window was hardcoded at 5s
(NORMAL_HEARTBEAT_TICKS). On oversubscribed hosts scheduling stalls
exceed it, so backups elect a new primary although the old one is
healthy, and client sessions die with the view change - loaded
parallel integration runs failed 23 message_retrieval tests this
way.

Surface the window as [cluster] heartbeat_timeout on the server-ng
config. ClusterConfig is forked into server_ng_config (like the
tcp/quic/websocket forks) because the legacy struct sources its
defaults from the legacy config.toml, which is frozen for ng-only
knobs. The runtime converts the duration to consensus ticks and
installs it before init, a validator enforces a 2s floor (the zero
sentinels are rejected too), and a unit test pins the TOML default
to the consensus built-in. The retrieval scenarios now run with a
60s window so box load cannot fake primary death.
@hubcio
hubcio dismissed stale reviews from mmodzelewski and numinnex via 7d511ee July 23, 2026 13:16
@hubcio
hubcio force-pushed the feat/surface-heartbeat-timeout branch from 73e9f9b to 7d511ee Compare July 23, 2026 13:16
@hubcio
hubcio merged commit 4c30a88 into master Jul 23, 2026
93 checks passed
@hubcio
hubcio deleted the feat/surface-heartbeat-timeout branch July 23, 2026 13:49
@github-actions github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants