Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(profiling): fix type check for exception types using isinstance #9551

Merged
merged 17 commits into from
Jun 20, 2024

Conversation

taegyunkim
Copy link
Contributor

@taegyunkim taegyunkim commented Jun 14, 2024

Previously, exc_type = <class Exception> would have failed the if check and now it passes as expected. This will result in showing exception type names in Timeline view in profiling, which is gated by a feature flag.

Tested using a sample application at https://github.com/DataDog/profiling-timeline-python-sample-app
before:
Screenshot 2024-06-18 at 5 32 14 PM

after:
Screenshot 2024-06-18 at 5 32 04 PM

Unfortunately, there's no easy way to write a Python unit test for this change. As the change touches a Cython file which should be a thin layer between a C++ extension and Python code. Also, we don't have any existing tests written in Cython that can be run using pytest, which could have been extended for this PR.

When DD_PROFILING_EXPORT_LIBDD_ENABLED is set and DD_PROFILING_STACK_V2_ENABLED is not set, the following chain of calls happens to collect exception information

  • ddtrace.profiling.collector.stack.StackCollector.collect_stack link
  • ddtrace.internal.datadog.profiling.ddup.SampleHandle.push_exceptioninfo link
  • ddup_push_exceptioninfo (extern C function) link
  • Datadog::Sample::push_exceptioninfo (C++ function) link

The only way to inspect the collected samples is using Datadog::Sample::profile_borrow() which isn't directly callable from a Python unit test.

Also, when STACK_V2 is enabled, this whole code path can be removed so I'd avoid spending too much time and effort in an interim solution, though I already did.

Checklist

  • Change(s) are motivated and described in the PR description
  • Testing strategy is described if automated tests are not included in the PR
  • Risks are described (performance impact, potential for breakage, maintainability)
  • Change is maintainable (easy to change, telemetry, documentation)
  • Library release note guidelines are followed or label changelog/no-changelog is set
  • Documentation is included (in-code, generated user docs, public corp docs)
  • Backport labels are set (if applicable)
  • If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Description motivates each change
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Change is maintainable (easy to change, telemetry, documentation)
  • Release note makes sense to a user of the library
  • Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@taegyunkim taegyunkim requested review from a team as code owners June 14, 2024 15:09
@taegyunkim taegyunkim marked this pull request as draft June 14, 2024 15:09
@taegyunkim taegyunkim changed the title fix(profiling): fix(profiling): print qualified names for exception types Jun 14, 2024
@taegyunkim taegyunkim changed the title fix(profiling): print qualified names for exception types fix(profiling): fix type check for exception types using isinstance, and print qualified names Jun 14, 2024
@datadog-dd-trace-py-rkomorn
Copy link

datadog-dd-trace-py-rkomorn bot commented Jun 14, 2024

Datadog Report

Branch report: taegyunkim/prof-9923-python-labels
Commit report: 27a7be2
Test service: dd-trace-py

✅ 0 Failed, 207 Passed, 837 Skipped, 48m 13.79s Total duration (29m 26.38s time saved)

@pr-commenter
Copy link

pr-commenter bot commented Jun 14, 2024

Benchmarks

Benchmark execution time: 2024-06-18 22:39:46

Comparing candidate commit 8f511bbeaedf9923207c6721f453e401899d6e3a in PR branch taegyunkim/prof-9923-python-labels with baseline commit 16f3f95 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 221 metrics, 9 unstable metrics.

@taegyunkim taegyunkim changed the title fix(profiling): fix type check for exception types using isinstance, and print qualified names fix(profiling): fix type check for exception types using isinstance Jun 14, 2024
@taegyunkim taegyunkim added the changelog/no-changelog A changelog entry is not required for this PR. label Jun 14, 2024
@taegyunkim taegyunkim marked this pull request as ready for review June 14, 2024 20:55
Copy link
Contributor

@danielsn danielsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a regression test for this?

Copy link
Contributor Author

I don't think we do. This is the commit/pr that introduced this if check.

interface.py looks like a test but I was not able to figure out how to invoke it.

@taegyunkim taegyunkim requested a review from a team as a code owner June 17, 2024 19:27
@taegyunkim taegyunkim force-pushed the taegyunkim/prof-9923-python-labels branch from e12105d to 8f511bb Compare June 18, 2024 21:34
@taegyunkim taegyunkim requested a review from danielsn June 18, 2024 21:34
Copy link
Contributor

@sanchda sanchda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM--thanks for fixing this

@taegyunkim taegyunkim enabled auto-merge (squash) June 20, 2024 13:38
@taegyunkim taegyunkim disabled auto-merge June 20, 2024 13:49
@taegyunkim taegyunkim force-pushed the taegyunkim/prof-9923-python-labels branch from 8f511bb to 27a7be2 Compare June 20, 2024 13:49
@taegyunkim
Copy link
Contributor Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented Jun 20, 2024

🚂 MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

Use /merge -c to cancel this operation!

@taegyunkim taegyunkim enabled auto-merge (squash) June 20, 2024 13:51
@taegyunkim taegyunkim merged commit f33856b into main Jun 20, 2024
75 checks passed
@taegyunkim taegyunkim deleted the taegyunkim/prof-9923-python-labels branch June 20, 2024 14:14
@dd-devflow
Copy link

dd-devflow bot commented Jun 20, 2024

🚂 MergeQueue: This merge request was already merged

This pull request was merged directly.

@taegyunkim taegyunkim added the Profiling Continous Profling label Jun 28, 2024
github-actions bot pushed a commit that referenced this pull request Jul 31, 2024
…9551)

Previously, `exc_type = <class Exception>` would have failed the if
check and now it passes as expected. This will result in showing
exception type names in Timeline view in profiling, which is gated by a
feature flag.

Tested using a sample application at
https://github.com/DataDog/profiling-timeline-python-sample-app
before:
<img width="291" alt="Screenshot 2024-06-18 at 5 32 14 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/72b8e897-9b1d-4238-a2d4-a77714493efd">

after:
<img width="286" alt="Screenshot 2024-06-18 at 5 32 04 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/ff572098-d3d5-48cc-8993-6d9ba1b30a33">

Unfortunately, there's no easy way to write a Python unit test for this
change. As the change touches a Cython file which should be a thin layer
between a C++ extension and Python code. Also, we don't have any
existing tests written in Cython that can be run using pytest, which
could have been extended for this PR.

When DD_PROFILING_EXPORT_LIBDD_ENABLED is set and
DD_PROFILING_STACK_V2_ENABLED is not set, the following chain of calls
happens to collect exception information
- ddtrace.profiling.collector.stack.StackCollector.collect_stack
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/profiling/collector/stack.pyx#L397)
-
ddtrace.internal.datadog.profiling.ddup.SampleHandle.push_exceptioninfo
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/ddup/_ddup.pyx#L257)
- ddup_push_exceptioninfo (extern C function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/interface.cpp#L238)
- Datadog::Sample::push_exceptioninfo (C++ function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/sample.cpp#L153)

The only way to inspect the collected samples is using
`Datadog::Sample::profile_borrow()` which isn't directly callable from a
Python unit test.

Also, when STACK_V2 is enabled, this whole code path can be removed so
I'd avoid spending too much time and effort in an interim solution,
though I already did.

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

(cherry picked from commit f33856b)
taegyunkim added a commit that referenced this pull request Aug 1, 2024
…9551)

Previously, `exc_type = <class Exception>` would have failed the if
check and now it passes as expected. This will result in showing
exception type names in Timeline view in profiling, which is gated by a
feature flag.

Tested using a sample application at
https://github.com/DataDog/profiling-timeline-python-sample-app
before:
<img width="291" alt="Screenshot 2024-06-18 at 5 32 14 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/72b8e897-9b1d-4238-a2d4-a77714493efd">

after:
<img width="286" alt="Screenshot 2024-06-18 at 5 32 04 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/ff572098-d3d5-48cc-8993-6d9ba1b30a33">

Unfortunately, there's no easy way to write a Python unit test for this
change. As the change touches a Cython file which should be a thin layer
between a C++ extension and Python code. Also, we don't have any
existing tests written in Cython that can be run using pytest, which
could have been extended for this PR.

When DD_PROFILING_EXPORT_LIBDD_ENABLED is set and
DD_PROFILING_STACK_V2_ENABLED is not set, the following chain of calls
happens to collect exception information
- ddtrace.profiling.collector.stack.StackCollector.collect_stack
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/profiling/collector/stack.pyx#L397)
-
ddtrace.internal.datadog.profiling.ddup.SampleHandle.push_exceptioninfo
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/ddup/_ddup.pyx#L257)
- ddup_push_exceptioninfo (extern C function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/interface.cpp#L238)
- Datadog::Sample::push_exceptioninfo (C++ function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/sample.cpp#L153)

The only way to inspect the collected samples is using
`Datadog::Sample::profile_borrow()` which isn't directly callable from a
Python unit test.

Also, when STACK_V2 is enabled, this whole code path can be removed so
I'd avoid spending too much time and effort in an interim solution,
though I already did.

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

(cherry picked from commit f33856b)
taegyunkim added a commit that referenced this pull request Aug 1, 2024
…backport 2.10] (#10013)

Backport f33856b from #9551 to 2.10.

Previously, `exc_type = <class Exception>` would have failed the if
check and now it passes as expected. This will result in showing
exception type names in Timeline view in profiling, which is gated by a
feature flag.

Tested using a sample application at
https://github.com/DataDog/profiling-timeline-python-sample-app
before:
<img width="291" alt="Screenshot 2024-06-18 at 5 32 14 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/72b8e897-9b1d-4238-a2d4-a77714493efd">

after: 
<img width="286" alt="Screenshot 2024-06-18 at 5 32 04 PM"
src="https://github.com/DataDog/dd-trace-py/assets/6655247/ff572098-d3d5-48cc-8993-6d9ba1b30a33">

Unfortunately, there's no easy way to write a Python unit test for this
change. As the change touches a Cython file which should be a thin layer
between a C++ extension and Python code. Also, we don't have any
existing tests written in Cython that can be run using pytest, which
could have been extended for this PR.

When DD_PROFILING_EXPORT_LIBDD_ENABLED is set and
DD_PROFILING_STACK_V2_ENABLED is not set, the following chain of calls
happens to collect exception information
- ddtrace.profiling.collector.stack.StackCollector.collect_stack
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/profiling/collector/stack.pyx#L397)
-
ddtrace.internal.datadog.profiling.ddup.SampleHandle.push_exceptioninfo
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/ddup/_ddup.pyx#L257)
- ddup_push_exceptioninfo (extern C function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/interface.cpp#L238)
- Datadog::Sample::push_exceptioninfo (C++ function)
[link](https://github.com/DataDog/dd-trace-py/blob/16f3f951addfad0ba3b65235e601a9c101f273c6/ddtrace/internal/datadog/profiling/dd_wrapper/src/sample.cpp#L153)

The only way to inspect the collected samples is using
`Datadog::Sample::profile_borrow()` which isn't directly callable from a
Python unit test.

Also, when STACK_V2 is enabled, this whole code path can be removed so
I'd avoid spending too much time and effort in an interim solution,
though I already did.

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: Taegyun Kim <taegyun.kim@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.10 changelog/no-changelog A changelog entry is not required for this PR. mergequeue-status: done Profiling Continous Profling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants