Skip to content

v0.23.13

Latest

Choose a tag to compare

@github-actions github-actions released this 27 Aug 08:12
· 2 commits to master since this release
b1766d9
fix(core): bound SSD prefetch staging allocations (#439)

## Summary

- Replace one whole-batch SSD prefetch slab per NUMA node with bounded
256 MiB staging chunks.
- Preserve per-slot NUMA placement, slot ordering, oversized-slot
handling, and the existing all-or-nothing batch allocation contract.
- Add a representative multi-group regression shape that verifies the
allocation bound.

## Production evidence before this change

Observed on one GB300 4-GPU node running a hybrid multi-group workload:

| Metric | Observed value |
| --- | ---: |
| SSD-prefetched data | 83 blocks / 8.52 GB |
| Largest contiguous staging request | 3.5 GiB per NUMA node |
| Cache blocks evicted | 4,608 |
| Logical cache bytes evicted | 415.6 GiB |
| Allocator bytes reclaimed | 462.8 GB |
| Pinned-pool usage | 484.5 GB -> 30.2 GB |

The allocator reclaims until its largest free extent can satisfy the
request. A multi-GiB whole-batch slab therefore turns fragmentation into
a large eviction burst. This is the SSD equivalent of the RDMA staging
issue addressed by #387.

## Allocation-shape A/B

The deterministic multi-group regression uses 56 blocks with eight 16
MiB slots split evenly across two NUMA nodes:

| Allocation shape | Baseline | This PR |
| --- | ---: | ---: |
| Bytes staged per NUMA | 3.5 GiB | 3.5 GiB |
| Maximum contiguous allocation | 3.5 GiB | 256 MiB |
| Allocations per NUMA | 1 | 14 |
| Slots preserved | 224 | 224 |

This establishes the allocation bound. Runtime eviction and hit-ratio
impact still need to be re-measured after deployment.

## Validation

- `cargo fmt --all -- --check`
- `cargo clippy -p pegaflow-core --all-targets --no-default-features
--features cuda-13,rdma -- -D warnings`
- `cargo test -p pegaflow-core --no-default-features --features
cuda-13,rdma -- --test-threads=1` (154 passed, 1 existing GPU-only test
ignored)
- `prek run`

---------

Signed-off-by: xiaguan <751080330@qq.com>