feat(openai): add `model` tag for openai metrics for consistency with openai saas integration #10282

lievan · 2024-08-19T19:07:58Z

Metrics from the OpenAI SaaS integration (openai.api.usage...) have a model tag. We should also tag metrics from OpenAI APM Integration with model for convenience (for example if customers want to use both openai.api.usage... metrics and openai.ratelimit... metrics in the same dashboard).

Addresses #10185

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2024-08-19T19:08:27Z

CODEOWNERS have been resolved as:

releasenotes/notes/openai-model-tag-2482b3d5b2905db9.yaml               @DataDog/apm-python
ddtrace/llmobs/_integrations/openai.py                                  @DataDog/ml-observability
tests/contrib/openai/test_openai_v0.py                                  @DataDog/ml-observability
tests/contrib/openai/test_openai_v1.py                                  @DataDog/ml-observability

datadog-dd-trace-py-rkomorn · 2024-08-19T19:23:55Z

Datadog Report

Branch report: evan.li/openai-model-tag
Commit report: 5bc8bbd
Test service: dd-trace-py

✅ 0 Failed, 2500 Passed, 38487 Skipped, 28m 52.93s Total Time

pr-commenter · 2024-08-19T21:12:14Z

Benchmarks

Benchmark execution time: 2024-08-21 18:16:41

Comparing candidate commit 5bc8bbd in PR branch evan.li/openai-model-tag with baseline commit 4d1e946 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 353 metrics, 47 unstable metrics.

lievan · 2024-08-21T17:39:52Z

/merge

dd-devflow · 2024-08-21T17:39:59Z

🚂 MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

Use /merge -c to cancel this operation!

dd-devflow · 2024-08-21T18:03:22Z

🚂 MergeQueue: queue is disabled

Added to the queue but the mergequeue is not enabled for now.

Use /merge -c to cancel this operation!

dd-devflow · 2024-08-22T13:34:56Z

🚂 MergeQueue: This merge request was already merged

This pull request was merged directly.

…h openai saas integration (#10282) Metrics from the OpenAI SaaS integration `(openai.api.usage...)` have a `model` tag. We should also tag metrics from OpenAI APM Integration with `model` for convenience (for example if customers want to use both `openai.api.usage...` metrics and `openai.ratelimit...` metrics in the same dashboard). Addresses #10185 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) --------- Co-authored-by: lievan <evan.li@datadoqhq.com> (cherry picked from commit 07ecb27)

…h openai saas integration [backport 2.12] (#10384) Backport 07ecb27 from #10282 to 2.12. Metrics from the OpenAI SaaS integration `(openai.api.usage...)` have a `model` tag. We should also tag metrics from OpenAI APM Integration with `model` for convenience (for example if customers want to use both `openai.api.usage...` metrics and `openai.ratelimit...` metrics in the same dashboard). Addresses #10185 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: lievan <42917263+lievan@users.noreply.github.com>

…h openai saas integration [backport 2.11] (#10382) Backport 07ecb27 from #10282 to 2.11. Metrics from the OpenAI SaaS integration `(openai.api.usage...)` have a `model` tag. We should also tag metrics from OpenAI APM Integration with `model` for convenience (for example if customers want to use both `openai.api.usage...` metrics and `openai.ratelimit...` metrics in the same dashboard). Addresses #10185 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: lievan <42917263+lievan@users.noreply.github.com> Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>

…h openai saas integration [backport 2.10] (#10383) Backport 07ecb27 from #10282 to 2.10. Metrics from the OpenAI SaaS integration `(openai.api.usage...)` have a `model` tag. We should also tag metrics from OpenAI APM Integration with `model` for convenience (for example if customers want to use both `openai.api.usage...` metrics and `openai.ratelimit...` metrics in the same dashboard). Addresses #10185 ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) Co-authored-by: lievan <42917263+lievan@users.noreply.github.com> Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>

lievan added 2 commits August 19, 2024 14:32

add model tag for openai integration

ede97c6

release note

773e6dd

lievan added 3 commits August 19, 2024 15:57

fix log test

8c72a8f

fix log tests

46fc181

rm model tag in tests for log tests

a8d01f3

lievan marked this pull request as ready for review August 19, 2024 22:07

lievan requested review from a team as code owners August 19, 2024 22:07

lievan requested review from mabdinur and quinna-h August 19, 2024 22:07

sabrenner approved these changes Aug 20, 2024

View reviewed changes

quinna-h approved these changes Aug 20, 2024

View reviewed changes

Merge branch 'main' into evan.li/openai-model-tag

5bc8bbd

dd-devflow bot added mergequeue-status: waiting mergequeue-status: queued and removed mergequeue-status: waiting labels Aug 21, 2024

lievan merged commit 07ecb27 into main Aug 22, 2024
157 of 159 checks passed

lievan deleted the evan.li/openai-model-tag branch August 22, 2024 13:34

dd-devflow bot added mergequeue-status: done and removed mergequeue-status: queued labels Aug 22, 2024

Yun-Kim added backport 2.10 backport 2.11 backport 2.12 labels Aug 26, 2024

Yun-Kim changed the title ~~chore(openai): add model tag for openai metrics for consistency with openai saas integration~~ feat(openai): add model tag for openai metrics for consistency with openai saas integration Sep 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(openai): add `model` tag for openai metrics for consistency with openai saas integration #10282

feat(openai): add `model` tag for openai metrics for consistency with openai saas integration #10282

lievan commented Aug 19, 2024 •

edited

Loading

github-actions bot commented Aug 19, 2024

datadog-dd-trace-py-rkomorn bot commented Aug 19, 2024 •

edited

Loading

pr-commenter bot commented Aug 19, 2024 •

edited

Loading

lievan commented Aug 21, 2024

dd-devflow bot commented Aug 21, 2024

dd-devflow bot commented Aug 21, 2024

dd-devflow bot commented Aug 22, 2024

feat(openai): add model tag for openai metrics for consistency with openai saas integration #10282

feat(openai): add model tag for openai metrics for consistency with openai saas integration #10282

Conversation

lievan commented Aug 19, 2024 • edited Loading

Checklist

Reviewer Checklist

github-actions bot commented Aug 19, 2024

datadog-dd-trace-py-rkomorn bot commented Aug 19, 2024 • edited Loading

Datadog Report

pr-commenter bot commented Aug 19, 2024 • edited Loading

Benchmarks

lievan commented Aug 21, 2024

dd-devflow bot commented Aug 21, 2024

dd-devflow bot commented Aug 21, 2024

dd-devflow bot commented Aug 22, 2024

feat(openai): add `model` tag for openai metrics for consistency with openai saas integration #10282

feat(openai): add `model` tag for openai metrics for consistency with openai saas integration #10282

lievan commented Aug 19, 2024 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Aug 19, 2024 •

edited

Loading

pr-commenter bot commented Aug 19, 2024 •

edited

Loading