Skip to content

Align cross-compile matrix with build_daemon metadata#2045

Merged
oferchen merged 1 commit into
masterfrom
fix-ci-validation-test-failures
Nov 7, 2025
Merged

Align cross-compile matrix with build_daemon metadata#2045
oferchen merged 1 commit into
masterfrom
fix-ci-validation-test-failures

Conversation

@oferchen
Copy link
Copy Markdown
Owner

@oferchen oferchen commented Nov 7, 2025

Summary

  • add explicit build_daemon flags to each Linux, macOS, and Windows cross-compile matrix entry
  • ensure CI metadata matches the expectations enforced by xtask validators

Testing

  • cargo test -p xtask validate_ci_cross_compile_matrix_accepts_workspace_configuration
  • cargo test -p xtask validate_documents_accepts_workspace_branding

Codex Task

@oferchen oferchen merged commit eaac0b4 into master Nov 7, 2025
@oferchen oferchen deleted the fix-ci-validation-test-failures branch November 7, 2025 20:55
@oferchen oferchen mentioned this pull request May 1, 2026
Merged
4 tasks
oferchen added a commit that referenced this pull request May 1, 2026
@oferchen
docs(fast_io): adaptive sizing audit + miss-rate telemetry (#1834, #2045
0c7ca46 
) (#3499)

Add an audit doc designing EMA-driven adaptive sizing for the io_uring
registered buffer pool, plus the lightweight telemetry counters that
the design depends on.

The audit (docs/audits/io-uring-adaptive-buffer-sizing.md) covers:

- Where the current sizing constants live (IoUringConfig defaults,
  for_large_files, for_small_files) and how they reach the pool.
- Where to measure miss rate without a syscall on the hot path.
- Proposed EMA design: alpha=0.2, WARMUP_SAMPLES=8, geometric grow /
  linear shrink, hysteresis cooldown, MIN/MAX clamps.
- Integration points: Phase 1 (telemetry) lands now; Phase 2
  (AdaptiveBufferSizer + resize driver) follows.
- Risks: registered-buffer reallocation cost, RLIMIT_MEMLOCK, the
  1024-buffer kernel cap, Drop ordering, in-flight SQEs, SQPOLL,
  per-checkout fetch_add overhead.

Telemetry (Phase 1):

- RegisteredBufferGroup carries total_acquires / total_misses
  AtomicU64 counters bumped inside checkout(). Cost is two Relaxed
  fetch_add per call; sub-percent at line rate.
- A new RegisteredBufferStats snapshot type with miss_rate(), modeled
  on BufferPoolStats in the engine.
- RegisteredBufferGroup::stats() accessor.
- The stub module mirrors the API so adaptive-sizing logic compiles
  on non-Linux targets.
- Five new unit tests cover the zero state, hit path, miss path,
  return-does-not-decrement invariant, and the helper math.

Phase 2 (the actual AdaptiveBufferSizer + resize cycle) is intentionally
not in this PR. The doc is the contract.
oferchen added a commit that referenced this pull request May 5, 2026
@oferchen
docs(fast_io): adaptive sizing audit + miss-rate telemetry (#1834, #2045
14538bc 
) (#3499)

Add an audit doc designing EMA-driven adaptive sizing for the io_uring
registered buffer pool, plus the lightweight telemetry counters that
the design depends on.

The audit (docs/audits/io-uring-adaptive-buffer-sizing.md) covers:

- Where the current sizing constants live (IoUringConfig defaults,
  for_large_files, for_small_files) and how they reach the pool.
- Where to measure miss rate without a syscall on the hot path.
- Proposed EMA design: alpha=0.2, WARMUP_SAMPLES=8, geometric grow /
  linear shrink, hysteresis cooldown, MIN/MAX clamps.
- Integration points: Phase 1 (telemetry) lands now; Phase 2
  (AdaptiveBufferSizer + resize driver) follows.
- Risks: registered-buffer reallocation cost, RLIMIT_MEMLOCK, the
  1024-buffer kernel cap, Drop ordering, in-flight SQEs, SQPOLL,
  per-checkout fetch_add overhead.

Telemetry (Phase 1):

- RegisteredBufferGroup carries total_acquires / total_misses
  AtomicU64 counters bumped inside checkout(). Cost is two Relaxed
  fetch_add per call; sub-percent at line rate.
- A new RegisteredBufferStats snapshot type with miss_rate(), modeled
  on BufferPoolStats in the engine.
- RegisteredBufferGroup::stats() accessor.
- The stub module mirrors the API so adaptive-sizing logic compiles
  on non-Linux targets.
- Five new unit tests cover the zero state, hit path, miss path,
  return-does-not-decrement invariant, and the helper math.

Phase 2 (the actual AdaptiveBufferSizer + resize cycle) is intentionally
not in this PR. The doc is the contract.
This was referenced May 5, 2026
Merged
Merged
oferchen added a commit that referenced this pull request May 5, 2026
@oferchen
docs(design): adaptive sizing for io_uring registered buffer pool (#2045
d4db245 
) (#3708)

Phase 2 design for the adaptive sizer that consumes the acquire/miss
telemetry already shipped on RegisteredBufferGroup. Mirrors the engine
BufferPool grow/shrink policy, with hysteresis and a quiescence guard
appropriate for a kernel-side resource.
oferchen added a commit that referenced this pull request May 5, 2026
@oferchen
docs(design): bound io_uring bgid u16 namespace exhaustion (#2044) (#…
8e526f2 
…3711)

Document the 16-bit io_uring buffer-group-id namespace surface, the
absence of an in-tree allocator today, theoretical and practical
exhaustion bounds, a free-list recycler proposal with generation
guard, hard-cap fallback to non-PBUF_RING I/O, and the unit and
integration test plan. Cross-references #2045 adaptive sizing as the
orthogonal per-ring concern.
@oferchen oferchen mentioned this pull request May 5, 2026
Merged
4 tasks
oferchen added a commit that referenced this pull request May 6, 2026
@oferchen
docs(design): io_uring submission-modes benchmark plan (#1626)
b8e35e0 
Plan benchmarking regular submit_and_wait vs SQPOLL vs std I/O across
small/large/mid file workloads, the CAP_SYS_NICE privilege matrix
including unprivileged container (#1624), and kernels 5.6 / 5.11 / 5.15
/ 6.0 / 6.6. Cross-references the session ring pool (#1408 / #1409),
PBUF_RING audit (#2043), and registered-buffer adaptive sizing (#2045).
oferchen added a commit that referenced this pull request May 6, 2026
@oferchen
docs(design): io_uring submission-modes benchmark plan (#1626) (#3718)
da465bf 
Plan benchmarking regular submit_and_wait vs SQPOLL vs std I/O across
small/large/mid file workloads, the CAP_SYS_NICE privilege matrix
including unprivileged container (#1624), and kernels 5.6 / 5.11 / 5.15
/ 6.0 / 6.6. Cross-references the session ring pool (#1408 / #1409),
PBUF_RING audit (#2043), and registered-buffer adaptive sizing (#2045).
@oferchen oferchen mentioned this pull request May 7, 2026
Merged
3 tasks
oferchen added a commit that referenced this pull request May 7, 2026
@oferchen
docs(audits): registered-buffer adaptive sizing follow-up (#2045) (#3879
02eb0bc 
)

Documents the static sizing of the io_uring registered-buffer pool,
the silent fallback to non-registered opcodes when slots are exhausted
under sustained queue depth, and a grow/shrink heuristic that mirrors
the engine BufferPool adaptive resizer (#1638) while accounting for
the syscall and RLIMIT_MEMLOCK cost of re-registration.
This was referenced May 13, 2026
Merged
Merged
Merged
Merged
Merged
oferchen added a commit that referenced this pull request May 17, 2026
@oferchen
docs: add io_uring registered buffer adaptive sizing design (#2045) (#…
f0b37dd 
…4232)

Records the signal layer, kernel constraints, and concrete trigger
thresholds for adaptive sizing of the registered buffer pool under
sustained pressure. Covers gaps left by the existing companion
documents: CQE wait time, slot exhaustion counter, sustained
backpressure window, IORING_REGISTER_BUFFERS_UPDATE shrink fast
path, and cgroup v2 memory headroom. Recommends deferring the
implementation until the session ring pool (#1937) lands so the
sizer is tuned against the multi-ring workload that motivates it.
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs(design): adaptive sizing for io_uring registered buffer pool (#2045
6787cdd 
) (#3708)

Phase 2 design for the adaptive sizer that consumes the acquire/miss
telemetry already shipped on RegisteredBufferGroup. Mirrors the engine
BufferPool grow/shrink policy, with hysteresis and a quiescence guard
appropriate for a kernel-side resource.
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs(design): bound io_uring bgid u16 namespace exhaustion (#2044) (#…
3b2f63b 
…3711)

Document the 16-bit io_uring buffer-group-id namespace surface, the
absence of an in-tree allocator today, theoretical and practical
exhaustion bounds, a free-list recycler proposal with generation
guard, hard-cap fallback to non-PBUF_RING I/O, and the unit and
integration test plan. Cross-references #2045 adaptive sizing as the
orthogonal per-ring concern.
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs(design): io_uring submission-modes benchmark plan (#1626) (#3718)
378f9e1 
Plan benchmarking regular submit_and_wait vs SQPOLL vs std I/O across
small/large/mid file workloads, the CAP_SYS_NICE privilege matrix
including unprivileged container (#1624), and kernels 5.6 / 5.11 / 5.15
/ 6.0 / 6.6. Cross-references the session ring pool (#1408 / #1409),
PBUF_RING audit (#2043), and registered-buffer adaptive sizing (#2045).
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs(audits): registered-buffer adaptive sizing follow-up (#2045) (#3879
567d3e3 
)

Documents the static sizing of the io_uring registered-buffer pool,
the silent fallback to non-registered opcodes when slots are exhausted
under sustained queue depth, and a grow/shrink heuristic that mirrors
the engine BufferPool adaptive resizer (#1638) while accounting for
the syscall and RLIMIT_MEMLOCK cost of re-registration.
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs: add io_uring registered buffer adaptive sizing design (#2045) (#…
bc3f2ca 
…4232)

Records the signal layer, kernel constraints, and concrete trigger
thresholds for adaptive sizing of the registered buffer pool under
sustained pressure. Covers gaps left by the existing companion
documents: CQE wait time, slot exhaustion counter, sustained
backpressure window, IORING_REGISTER_BUFFERS_UPDATE shrink fast
path, and cgroup v2 memory headroom. Recommends deferring the
implementation until the session ring pool (#1937) lands so the
sizer is tuned against the multi-ring workload that motivates it.
oferchen added a commit that referenced this pull request May 18, 2026
@oferchen
docs: add io_uring registered buffer adaptive sizing design (#2045) (#…
8e7b873 
…4232)

Records the signal layer, kernel constraints, and concrete trigger
thresholds for adaptive sizing of the registered buffer pool under
sustained pressure. Covers gaps left by the existing companion
documents: CQE wait time, slot exhaustion counter, sustained
backpressure window, IORING_REGISTER_BUFFERS_UPDATE shrink fast
path, and cgroup v2 memory headroom. Recommends deferring the
implementation until the session ring pool (#1937) lands so the
sizer is tuned against the multi-ring workload that motivates it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@oferchen