Skip to content

[codex-analytics] ingest server requests and responses#17088

Merged
rhan-oai merged 1 commit intomainfrom
pr17088
Apr 29, 2026
Merged

[codex-analytics] ingest server requests and responses#17088
rhan-oai merged 1 commit intomainfrom
pr17088

Conversation

@rhan-oai
Copy link
Copy Markdown
Collaborator

@rhan-oai rhan-oai commented Apr 8, 2026

Why

Codex analytics needs a typed seam for app-server-originated request/response traffic so future tool-approval analytics can consume those facts without adding bespoke callsite tracking each time. Server responses arrive as JSON-RPC id + result payloads, so analytics has to reconstruct the matching typed response from the original typed request while that request context still exists in app-server.

This also puts analytics on the app-server outbound path, which needs to avoid keeping the runtime alive during shutdown. The final ownership fix keeps the normal strong auth-manager retention in analytics and makes the external-auth refresh bridge hold a weak back-reference to OutgoingMessageSender, breaking the runtime cycle at the bridge boundary instead of exposing retention policy through the analytics client API.

What changed

  • Adds typed ServerRequest and ServerResponse analytics facts, plus AnalyticsEventsClient::track_server_request and track_server_response.
  • Renames the existing client-side facts to ClientRequest and ClientResponse so reducers can distinguish client-to-server traffic from server-to-client traffic.
  • Adds ServerRequest::response_from_result, allowing a stored typed request to decode the matching typed server response from a raw JSON-RPC result payload.
  • Threads AnalyticsEventsClient through OutgoingMessageSender and records targeted server requests, replayed targeted requests, and matching targeted responses with the responding connection id needed for correlation.
  • Intentionally leaves broadcast server requests/responses out of analytics for now because the current model is per connection, while broadcasts fan one logical request out across multiple connections.
  • Breaks the app-server shutdown cycle by storing Weak<OutgoingMessageSender> in ExternalAuthRefreshBridge and upgrading it only when an external-auth refresh is actually requested.
  • Keeps reducer ingestion of the new server-side facts as no-ops for now; this PR is plumbing for later tool-approval analytics work.

Verification

  • cargo test -p codex-analytics
  • cargo test -p codex-app-server outgoing_message::tests::
    • Covers typed-response reconstruction plus the targeted, replayed, broadcast-exclusion, and response-attribution analytics paths.

Follow-up

This PR intentionally stops at ingestion plumbing, so ServerRequest and ServerResponse facts are still reducer no-ops. Once a follow-up PR adds real downstream analytics output for those facts:

  • replace the temporary pre-reducer observation seam with reducer tests for the emitted event shape;
  • add end-to-end coverage in app-server/tests/suite/v2/analytics.rs for the real app-server workflow and captured analytics payload;
  • remove the temporary sender-level observer tests added here in favor of the real-output coverage above.

Stack created with Sapling. Best reviewed with ReviewStack.

Comment thread codex-rs/app-server/src/in_process.rs Outdated
let (outgoing_tx, mut outgoing_rx) = mpsc::channel::<OutgoingEnvelope>(channel_capacity);
let outgoing_message_sender = Arc::new(OutgoingMessageSender::new(outgoing_tx));
let auth_manager =
AuthManager::shared_from_config(args.config.as_ref(), args.enable_codex_api_key_env);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we have an auth_manager on InProcessStartArgs?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right now is not the case

pub struct InProcessStartArgs {

@rhan-oai rhan-oai force-pushed the pr17088 branch 3 times, most recently from 32d2d05 to 840f271 Compare April 22, 2026 19:58
@rhan-oai
Copy link
Copy Markdown
Collaborator Author

@codex

chatgpt-codex-connector[bot]

This comment was marked as resolved.

@openai openai deleted a comment from chatgpt-codex-connector Bot Apr 23, 2026
@openai openai deleted a comment from chatgpt-codex-connector Bot Apr 23, 2026
@openai openai deleted a comment from chatgpt-codex-connector Bot Apr 24, 2026
@rhan-oai
Copy link
Copy Markdown
Collaborator Author

@codex

@rhan-oai
Copy link
Copy Markdown
Collaborator Author

@codex

Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 98f89b7ea8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/analytics/src/client.rs Outdated
Comment on lines +60 to +67
let auth_manager = Arc::downgrade(&auth_manager);
tokio::spawn(async move {
let mut reducer = AnalyticsReducer::default();
while let Some(input) = receiver.recv().await {
let mut events = Vec::new();
reducer.ingest(input, &mut events).await;
let Some(auth_manager) = auth_manager.upgrade() else {
break;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain AuthManager in analytics worker

AnalyticsEventsQueue::new downgrades auth_manager to Weak and exits the worker when upgrade fails. Because AnalyticsEventsClient::new no longer retains a strong AuthManager, callers that pass a temporary Arc will silently stop analytics processing after startup. This is a behavioral regression from the previous ownership model and can drop all later events.

Useful? React with 👍 / 👎.

@openai openai deleted a comment from chatgpt-codex-connector Bot Apr 27, 2026
@rhan-oai rhan-oai requested a review from a team as a code owner April 27, 2026 23:39
@rhan-oai
Copy link
Copy Markdown
Collaborator Author

@codex

@chatgpt-codex-connector
Copy link
Copy Markdown
Contributor

Codex Review: Didn't find any major issues. Breezy!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@openai openai deleted a comment from chatgpt-codex-connector Bot Apr 28, 2026
Comment thread codex-rs/analytics/src/client.rs Outdated
}

#[derive(Debug, Clone, Copy)]
pub enum AuthManagerRetention {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something doesn't feel right here.

Copy link
Copy Markdown
Collaborator

@owenlin0 owenlin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: perhaps call codex-rs/app-server/src/analytics_events.rs -> analytics_utils.py?

[followup?] also, now we decode the same JSON payload twice in the happy path: once for analytics, then again in the actual handler. as a followup it'd be nice to see if there's a way we can decode the JSON payload just once (probably not a big deal atm in practice since the client likely doesn't send massive JSON responses - the only use case where that should be possible is dynamic tool calls)

@rhan-oai rhan-oai enabled auto-merge (squash) April 29, 2026 19:03
@rhan-oai rhan-oai merged commit 0690ab0 into main Apr 29, 2026
53 of 55 checks passed
@rhan-oai rhan-oai deleted the pr17088 branch April 29, 2026 19:56
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants