perf: benchmark OMP Codex transports#49
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an OMP Codex transport benchmark harness (WebSocket vs SSE) to support measuring and reporting transport performance/diagnostics without handling credentials inside T4 Code, along with documentation and CI-integrated tests.
Changes:
- Introduces a Node CLI harness to run paired/alternating OMP benchmark runs and produce a redacted JSON report (timings, tokens, tool-call counts, transport diagnostics).
- Adds focused unit tests for argument parsing, prompt construction, event validation, aggregation, and secret-safe error reporting.
- Documents measured results, reproduction steps, and the benchmark’s proof boundaries.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| scripts/benchmark-omp-codex-transport.mjs | New benchmark harness + JSON parsing/aggregation and transport diagnostics collection. |
| scripts/benchmark-omp-codex-transport.test.mjs | New unit tests covering parsing, validation, aggregation, and bounded error messaging. |
| package.json | Wires the new benchmark test into the existing test:tooling suite. |
| docs/OMP_CODEX_TRANSPORT_BENCHMARK.md | Documents methodology, results, reproduction steps, and proof boundary. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b5e54dbc9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Summary
Verification