Skip to content

Get benchmark node memory stats from /proc#7856

Merged
achamayou merged 5 commits intomicrosoft:mainfrom
eddyashton:get_memory_stats_from_proc
May 1, 2026
Merged

Get benchmark node memory stats from /proc#7856
achamayou merged 5 commits intomicrosoft:mainfrom
eddyashton:get_memory_stats_from_proc

Conversation

@eddyashton
Copy link
Copy Markdown
Member

Follow-up to #7822. We no longer need a built-in /node/memory endpoint, since these stats can be gathered for current platforms from standard tools. This uses those standard tools (reading from /proc/{pid}/status) to get memory use during our e2e benchmark runs, restoring those values going to Bencher.

Copilot AI review requested due to automatic review settings April 30, 2026 10:37
@eddyashton eddyashton requested a review from a team as a code owner April 30, 2026 10:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces the removed /node/memory endpoint by collecting node memory metrics directly from /proc/<pid>/status during e2e benchmark runs (and including per-test memory snapshots in the e2e suite report), so these values can continue to be published to Bencher on supported platforms.

Changes:

  • Add /proc/<pid>/status parsing helper to extract RSS/peak RSS/virtual size in bytes.
  • Publish primary-node memory metrics to Bencher in benchmark runners.
  • Record per-node memory stats in e2e_suite.py test run reporting.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/infra/proc.py Adds get_proc_memory_stats() to read RSS/HWM/VmSize from /proc.
tests/infra/runner.py Collects primary PID memory stats and publishes them to Bencher after benchmark runs.
tests/infra/piccolo_driver.py Collects primary PID memory stats and publishes them to Bencher after benchmark runs.
tests/infra/basicperf.py Collects primary PID memory stats and publishes them to Bencher (when primary wasn’t stopped).
tests/e2e_suite.py Adds mem_stats() and includes memory info in per-test JSON reporting.

Comment thread tests/infra/piccolo_driver.py Outdated
Comment thread tests/infra/basicperf.py Outdated
Comment thread tests/e2e_suite.py Outdated
Comment thread tests/infra/runner.py Outdated
@achamayou achamayou merged commit 4255f89 into microsoft:main May 1, 2026
19 checks passed
cjen1-msft pushed a commit to cjen1-msft/CCF that referenced this pull request May 6, 2026
Co-authored-by: Amaury Chamayou <amchamay@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants