Skip to content

Conversation

@lievan
Copy link
Contributor

@lievan lievan commented May 20, 2025

Swap error message & type for the openai agents sdk

The error data returned by openai is currently being set to the error type on the llm obs span. Since its verbose, this has caused some UX issues since error types are expected to be short.

The error "message" from oai agents tracing is actually more fit to be our error type, since its a concise message

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@lievan lievan marked this pull request as ready for review May 20, 2025 15:13
@lievan lievan requested review from a team as code owners May 20, 2025 15:13
@lievan lievan requested review from ZStriker19 and erikayasuda May 20, 2025 15:13
@github-actions
Copy link
Contributor

CODEOWNERS have been resolved as:

releasenotes/notes/change-oai-error-e7b24f61c1cfdc83.yaml               @DataDog/apm-python
ddtrace/llmobs/_integrations/openai_agents.py                           @DataDog/ml-observability
tests/contrib/openai_agents/test_openai_agents_llmobs.py                @DataDog/ml-observability

@github-actions
Copy link
Contributor

github-actions bot commented May 20, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 231 ± 2 ms.

The average import time from base is: 233 ± 2 ms.

The import time difference between this PR and base is: -2.33 ± 0.1 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 2.006 ms (0.87%)
ddtrace.bootstrap.sitecustomize 1.301 ms (0.56%)
ddtrace.bootstrap.preload 1.301 ms (0.56%)
ddtrace.internal.remoteconfig.client 0.631 ms (0.27%)
ddtrace 0.705 ms (0.31%)
ddtrace._logger 0.034 ms (0.01%)
logging 0.034 ms (0.01%)
traceback 0.034 ms (0.01%)
contextlib 0.034 ms (0.01%)
ddtrace.internal._unpatched 0.023 ms (0.01%)

@pr-commenter
Copy link

pr-commenter bot commented May 20, 2025

Benchmarks

Benchmark execution time: 2025-05-29 15:47:11

Comparing candidate commit 1b2813a in PR branch evan.li/change-error-oai-agents with baseline commit 074b3be in branch main.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 504 metrics, 9 unstable metrics.

scenario:iastaspects-lstrip_aspect

  • 🟥 execution_time [+1.342µs; +1.532µs] or [+10.286%; +11.738%]

scenario:iastaspects-strip_aspect

  • 🟥 execution_time [+1.122µs; +1.437µs] or [+8.614%; +11.029%]

scenario:iastaspectsospath-ospathsplit_aspect

  • 🟥 execution_time [+815.245ns; +937.759ns] or [+16.679%; +19.185%]

Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>
@lievan lievan enabled auto-merge (squash) May 23, 2025 14:51
@lievan lievan merged commit 6c6f6aa into main May 29, 2025
352 of 354 checks passed
@lievan lievan deleted the evan.li/change-error-oai-agents branch May 29, 2025 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants