Fix openAI-compatible telemetry response attributes use the request namespace type - #7127
Conversation
fubhy
added
the
audit
🦋 Changeset detectedLatest commit: 3aa0b3b The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The reproduction test correctly captures the bug, but the implementation fix in packages/ai/openai-compat/src/OpenAiTelemetry.ts is missing. CI will fail and the PR title's "Fix" claim is unfulfilled until the source type is corrected.
Reviewed changes
This PR adds a TSTyche type test in packages/ai/openai-compat/typetest/OpenAiTelemetry.tst.ts that verifies OpenAiTelemetryAttributes accepts gen_ai.openai.response.system_fingerprint and rejects the gen_ai.openai.request.system_fingerprint namespace.
⚠️ Implementation fix is missing
The test reproduces the bug as intended, but packages/ai/openai-compat/src/OpenAiTelemetry.ts was not modified. Line 30 still maps ResponseAttributes under the request namespace:
& Telemetry.AttributesWithPrefix<ResponseAttributes, "gen_ai.openai.request">The runtime helper on line 125 already emits gen_ai.openai.response.*, so the type declaration should match:
& Telemetry.AttributesWithPrefix<ResponseAttributes, "gen_ai.openai.response">Please apply the source fix to the same branch and confirm pnpm test-types packages/ai/openai-compat/typetest/OpenAiTelemetry.tst.ts passes.
Technical details
# Missing response namespace fix
## Affected sites
- packages/ai/openai-compat/src/OpenAiTelemetry.ts:30 — `ResponseAttributes` uses `"gen_ai.openai.request"` instead of `"gen_ai.openai.response"`
## Required outcome
- `OpenAiTelemetryAttributes` exposes `gen_ai.openai.response.system_fingerprint`
- `OpenAiTelemetryAttributes` rejects `gen_ai.openai.request.system_fingerprint`
- `pnpm test-types packages/ai/openai-compat/typetest/OpenAiTelemetry.tst.ts` passes
## Suggested approach
Change line 30 of `src/OpenAiTelemetry.ts` to:
```ts
& Telemetry.AttributesWithPrefix<ResponseAttributes, "gen_ai.openai.response">
```@v0 or keep the SHA fresh with Dependabot ｜ Fix it ➔ ｜ View workflow run ｜ Using Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This incremental pass covered the new commit that completes the PR: the implementation fix and a patch changeset.
- Applied the response-namespace fix in
packages/ai/openai-compat/src/OpenAiTelemetry.ts—ResponseAttributesis now prefixed with"gen_ai.openai.response", matching the runtime helper. - Added a patch changeset for
@effect/ai-openai-compatdescribing the namespace fix. - The reproduction type test already present now passes, confirming
gen_ai.openai.response.system_fingerprintis accepted andgen_ai.openai.request.system_fingerprintis rejected.
ℹ️ Sibling package has the same copied-prefix issue
The same ResponseAttributes prefix bug appears to remain in packages/ai/openai/src/OpenAiTelemetry.ts, which is outside this PR's openai-compat scope. Worth tracking separately if it hasn't been fixed already.
@v0 or keep the SHA fresh with Dependabot ｜ View workflow run ｜ Using Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
There was a problem hiding this comment.
Important
The fix is correct, but the new commit deletes the regression test that guarded it, so the type-level bug this PR fixes can silently regress.
Reviewed changes
This incremental pass covers the single new commit since the prior approved review.
- Removed the reproduction type test
packages/ai/openai-compat/typetest/OpenAiTelemetry.tst.ts, which asserted thatOpenAiTelemetryAttributesacceptsgen_ai.openai.response.system_fingerprintand rejectsgen_ai.openai.request.system_fingerprint. The source fix itself is unchanged and still in place.
For reference, the fix (src/OpenAiTelemetry.ts:31) now has no automated guard — see the inline note.
Technical details
# Regression test for the namespace fix was removed
## Affected sites
- packages/ai/openai-compat/typetest/OpenAiTelemetry.tst.ts — deleted by commit 3aa0b3b5e1
## Required outcome
- Keep an autoregressive guard asserting the response namespace (`gen_ai.openai.response`) is accepted and the request namespace (`gen_ai.openai.request`) is rejected, so the fix in src/OpenAiTelemetry.ts:31 cannot silently regress.
## Open questions for the human
- If the removal was intentional (e.g. type tests for this shape are considered noise), it would help to say so; otherwise restore the test or move the coverage elsewhere.@v0 or keep the SHA fresh with Dependabot ｜ Fix it ➔ ｜ View workflow run ｜ Using DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
2 participants
Summary
Both aliases apply AttributesWithPrefix<ResponseAttributes, gen_ai.openai.request>, so they expose gen_ai.openai.request.system_fingerprint and reject gen_ai.openai.response.system_fingerprint. Both runtime helpers emit the latter.
Important
This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.
OpenAI-compatible telemetry response attributes use the request namespace type
Module:
packages/ai/openai-compat/src/OpenAiTelemetry.tsAudit ID:
relsem-openai-compat-telemetry-response-namespaceSeverity / confidence: medium / high
What happens
Both aliases apply AttributesWithPrefix<ResponseAttributes, gen_ai.openai.request>, so they expose gen_ai.openai.request.system_fingerprint and reject gen_ai.openai.response.system_fingerprint. Both runtime helpers emit the latter.
Why it happens
A copied request prefix is used for ResponseAttributes in both public type intersections.
Expected behavior
Provider response attributes represented at the type level use the same gen_ai.openai.response prefix emitted at runtime.
Relevant implementation
These links and excerpts are pinned to audit base
b206fa5d7655c1634c9993410a9203f6616a5ca2.packages/ai/openai-compat/src/OpenAiTelemetry.ts:1View problematic code at
packages/ai/openai-compat/src/OpenAiTelemetry.ts:1View exact lines on GitHub
Reproduction
Observed failure: Independently rerun; failed at the intended semantic assertion.
Implementation handoff
The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.
Audit provenance
b206fa5d7655c1634c9993410a9203f6616a5ca2b206fa5d7655c1634c9993410a9203f6616a5ca2relsem-openai-compat-telemetry-response-namespaceCloses EFF-561