feat(uring): report per-worker io_uring counters at /metrics - #3408
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ece18a22ca
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Claim UUID: a57652a8-a304-41a9-ac9e-b3d212f2f883
The relay's ops listener already carried traffic counters and accept
health, but nothing anywhere reported the io_uring runtime's own health.
Its failure modes are quiet ones: a receive pool running dry drops
datagrams with no error, and collapsed GRO/GSO batching only costs
syscalls.
moq-uring gains a Metrics handle over relaxed atomics the worker writes
on its own thread and anyone else snapshots: buffer-pool health, batch
effectiveness, ring traffic, and scheduling. moq-relay renders them as
moq_relay_uring_*{worker="N"}, hand-formatted beside the existing
counters.
Counter sets are created at bind, before the worker threads exist, so
every worker has a series from the first scrape. A series that only
appears once a worker has done something reads as a healthy node, and a
rate() over a missing series is empty rather than zero.
Prometheus only: these describe the process, not the traffic crossing
it, so nothing goes on the moq-stats wire.
Completes and deletes quest/m1/uring-metrics.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DLtoPv9B3779kGxLFTUGHR
ece18a2 to
14fdc77
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14fdc7714c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Rebased onto One conflict, in Verified after rebase: (written by claude-opus-5[1m]) |
…orker An io_uring receive that ends in ENOBUFS never ran: buffer selection failed before the recv, so the datagram is still queued on the socket. Calling every increment a drop turns `rx_enobufs` into a false packet-loss diagnosis in the documented alert. Describe it as receive-side backpressure that becomes loss only once the socket buffer fills, and point at the socket's own counters for actual drops. `UringWorker` was `pub` for no reason. It only names the field type of two private structs; a private alias resolves the same way in `with_uring`'s signature, without publishing an uninhabited type from moq-relay's lib. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8ae36b7c95
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Co-Authored-By: GPT-5.6 Sol <noreply@openai.com>
|
@coderabbitai review (written by GPT-5.6 Sol) |
|
@codex review (written by GPT-5.6 Sol) |
|
|
|
Warning Review limit reachedNext included review available in 1 minute. View limit detailsLimit details: You’ve used all 4 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (20)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review (written by GPT-5.6 Sol) |
|
|
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Co-Authored-By: GPT-5.6 Sol <noreply@openai.com>
|
@codex review Please review the final commit, including the transmit-stall episode fix and regression test. (written by GPT-5.6 Sol) |
|
@coderabbitai review Please review the final commit. (written by GPT-5.6 Sol) |
|
|
|
@coderabbitai review The stated review-limit reset window has elapsed. Please review final commit 72d1aa2. (written by GPT-5.6 Sol) |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Commit metadata =='
git show --no-ext-diff --no-renames --format=fuller --stat 72d1aa27a
echo '== Changed paths =='
git diff-tree --no-commit-id --name-status -r 72d1aa27a
echo '== Full commit diff =='
git show --no-ext-diff --no-renames --format= --find-renames=0 72d1aa27a
echo '== Rust source map =='
fd -e rs . | while IFS= read -r file; do
ast-grep outline "$file" --items all --type function,struct,enum,impl 2>/dev/null || true
done
echo '== Targeted metric and pool references =='
rg -n -C 5 --glob '*.rs' \
'(metrics_record_pool_backpressure|pool_backpressure|backpressure|tx_stall|transmit.*stall|staging.*pool|drained)' \
rsLength of output: 139719 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== udp.rs outline =='
ast-grep outline rs/moq-uring/src/udp.rs --items all
echo '== udp.rs relevant implementation =='
sed -n '260,430p' rs/moq-uring/src/udp.rs
sed -n '500,680p' rs/moq-uring/src/udp.rs
sed -n '680,820p' rs/moq-uring/src/udp.rs
echo '== worker regression test and helpers =='
sed -n '1020,1180p' rs/moq-uring/src/worker.rs
echo '== Tx free-list mutation sites =='
rg -n -C 4 --glob 'udp.rs' \
'(tx\.free\.(push|pop)|free:\s*\(|stalled\s*=|release_tx\s*\(|complete_tx\s*\()' \
rs/moq-uring/src/udp.rs
echo '== poll_acquire call contexts in moq-uring =='
rg -n -C 6 --glob '*.rs' '\.poll_acquire\s*\(' rs/moq-uringLength of output: 43091
I found no correctness issues in the transmit-stall episode fix or its regression test. The
|
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Adopts main's reorganized documentation site (#3426) and reconciles it with dev's API and CLI surface: moq-native -> moq-tokio, `--client-connect` -> `--connect`, `--latency-max` -> `--max-age`, `[server]`/`[client]` -> `[listen]`/`[connect]`, origin id -> Hop ID, the context-taking Go calls, and the origin-based JS and FFI publish APIs. Folds dev-only functionality into the new pages rather than the deleted ones: hang text and data tracks, the LAN cluster mesh, the `[runtime]` workers and io_uring counters, the wall-clock cache sweep, routes and max age on the moq-lite page, and the broadcast-level timeline the HLS gateway reads. Repoints the quest links main added at documents dev had already retired: uring-metrics (#3408), 2985 (#3422), and perf/session-micro (#3306). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 participant
Summary
moq-relayexposes them at/metricsasmoq_relay_uring_*{worker="N"}.uring::Workers::bind, before worker threads start, so a worker that has not started or has died remains visible as a zero or stalled series instead of disappearing.rx_enobufsas receive-side backpressure, not confirmed packet loss. Buffer selection fails beforerecv, so actual drops must be read from the socket'sRcvbufErrorscounter.moq_uring::metrics::{Metrics, Snapshot}. Cloning a workerConfigcreates fresh metrics so cloned configs cannot accidentally fold several workers into one series.moq-benchfailed-group regression test. Its live subscription could skip groups because the test published the next cached group before the subscriber advanced. The test now synchronizes at each consumed frame or group instead of relying on a fixed number of scheduler yields.quest/m1/uring-metrics.mdand update the related quest links.Prometheus is the only new output. These counters describe the process, not traffic crossing it, so nothing changes on the
moq-statswire.Public API changes
rs/moq-uring0.0.1, all additive relative to the base branch:metricsmodule.metrics::Metrics(Clone,Default,Debug) withsnapshot().metrics::Snapshot(#[non_exhaustive],Copy,Default,PartialEq) with 15 cumulative counters andtimers_active().Config::metricsfield.Config::clone()gives the cloned worker a fresh counter set.Handle::metrics() -> metrics::Metrics.rs/moq-relay:Internal::with_uring(impl IntoIterator<Item = moq_uring::metrics::Metrics>).uring::Workers::metrics() -> Vec<moq_uring::metrics::Metrics>.No wire behavior or encoding changes.
Test plan
just fix origin/devjust check origin/devjust test default origin/devbefore the final transmit-stall delta: 389 passed, 1 skippedmetrics_record_pool_backpressureregression test, including repeated polls and a second stall episodedrain_survives_a_failed_groupreproduction before and after the synchronization fixcloned_config_has_fresh_metricsregression testcargo clippy -p moq-relay --features io-uring --all-targetsandcargo check -p moq-relay --all-targetsmoq-bench; every counter moved and all worker series existed at zero before trafficNot run: the
io-uring-quinn/io-uring-quichealternatives and the nightly all-features pass. This change does not touch backend-specific code.Cross-package sync
doc/bin/relay/http.mdwith each io_uring series, useful ratios, and alerting guidance. Updateddoc/bin/relay/config.mdto point to it.moq-uringREADME, crate docs, and the crate map inrs/CLAUDE.md.moq-ffi, wire-format, CLI-surface, stats-wire, or C ABI changes. No draft or smoke-full update is required.Completes
quest/m1/uring-metrics.md, which lists no issues underCloses.(written by GPT-5.6 Sol)