feat: add GreenletProfiler #246

jaychoww · 2022-10-25T02:47:44Z

No description provided.

jaychoww · 2022-10-25T02:52:01Z

it is a draft, but it can profiling with threading model and greenlet model automatically now. it needs more things:
test cases

wu-sheng · 2022-10-25T03:28:10Z

Verifying through e2e profiling case would be a good place to start.

wu-sheng · 2022-10-25T03:33:07Z

Profiling e2e has been included in the main repo, https://github.com/apache/skywalking/blob/317d539371c588c4754029134ad76087ecb6d0cd/.github/workflows/skywalking.yaml#L435-L454 powered by our e2e-infra tool, https://skywalking.apache.org/docs/skywalking-infra-e2e/next/readme/

You could learn how we run tests using the latest agent with real OAP/SkyWalking backend, and verify the result through SkyWalking CLI through this tool.

jaychoww · 2022-10-25T03:37:26Z

thank you, it is the docs what I need

sonatype-lift · 2022-10-28T16:02:09Z

tests/e2e/base/provider/flask_provider.py

+
+if __name__ == '__main__':
+    # noinspection PyTypeChecker
+    app.run(host='0.0.0.0', port=9090)


hardcoded_bind_all_interfaces: Possible binding to all interfaces.

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

sonatype-lift · 2022-10-28T16:02:12Z

tests/e2e/case/profiling/greenlet/start_gevent.sh

@@ -0,0 +1,3 @@
+set -ex


SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

jaychoww · 2022-10-28T16:32:42Z

there is something wrong when profiling start:

skywalking [Thread-127] [DEBUG] profile task [1666974131787_ZTJlLXNlcnZpY2UtcHJvdmlkZXI=.1] for endpoint [/artist] started
Traceback (most recent call last):
  File "src/gevent/_waiter.py", line 122, in gevent._gevent_c_waiter.Waiter.switch
  File "/skywalking-python/venv/lib/python3.6/site-packages/apache_skywalking-0.8.0-py3.6.egg/skywalking/profile/profile_context.py", line 329, in callback
    agent.add_profiling_snapshot(snapshot)
  File "/skywalking-python/venv/lib/python3.6/site-packages/apache_skywalking-0.8.0-py3.6.egg/skywalking/agent/__init__.py", line 288, in add_profiling_snapshot
    __snapshot_queue.put(snapshot)
  File "/usr/local/lib/python3.6/queue.py", line 126, in put
    with self.not_full:
  File "/usr/local/lib/python3.6/threading.py", line 240, in __enter__
    return self._lock.__enter__()
  File "src/gevent/_semaphore.py", line 278, in gevent._gevent_c_semaphore.Semaphore.__enter__
  File "src/gevent/_semaphore.py", line 279, in gevent._gevent_c_semaphore.Semaphore.__enter__
  File "src/gevent/_semaphore.py", line 180, in gevent._gevent_c_semaphore.Semaphore.acquire
  File "/skywalking-python/venv/lib/python3.6/site-packages/gevent/thread.py", line 121, in acquire
    acquired = BoundedSemaphore.acquire(self, blocking, timeout)
  File "src/gevent/_semaphore.py", line 180, in gevent._gevent_c_semaphore.Semaphore.acquire
  File "src/gevent/_semaphore.py", line 249, in gevent._gevent_c_semaphore.Semaphore.acquire
  File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
  File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
  File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 64, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 67, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
  File "src/gevent/_greenlet_primitives.py", line 68, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
gevent.exceptions.BlockingSwitchOutError: Impossible to call blocking function in the event loop callback
2022-10-28T16:22:34Z <built-in method switch of gevent._gevent_cgreenlet.Greenlet object at 0x7ff213196a48> failed with BlockingSwitchOutError

this Exception will raise when gevent upgrade to version>1.4.0, and before the greenlet quit and switch to the hub.
so we can catch it for a signal to quit the profiling

tests/e2e/case/profiling/provider/provider.py

.github/workflows/CI.yaml

Superskyyy · 2022-11-02T15:54:08Z

Generally looks good to me. @kezhenxu94 can you take a look? I'm not very familiar with the profiling feature.

Superskyyy · 2022-11-04T17:12:37Z

The sonatype bug can be ignored, since it's testing code.

Superskyyy · 2022-11-04T17:17:07Z

We will need documentation. The original documentation for threading-profiling was posted to here https://skywalking.apache.org/blog/2021-09-12-skywalking-python-profiling/, now with the addition of Greenlet Profiler, it will be better to move it back to our repository and add the corresponding part for greenlet. @westarest, can you work on the docs in the next PR?

skywalking/profile/profile_context.py

jaychoww · 2022-11-05T03:34:29Z

We will need documentation. The original documentation for threading-profiling was posted to here https://skywalking.apache.org/blog/2021-09-12-skywalking-python-profiling/, now with the addition of Greenlet Profiler, it will be better to move it back to our repository and add the corresponding part for greenlet. @westarest, can you work on the docs in the next PR?

no problem. I can post a document about the difference between the Threading profiler and the Greenlet profiler that I committed.

kezhenxu94

Thanks!

wu-sheng

Cool. @westarest Would you like to write a new blog for this feature, to introduce why it is important, and how you implement this, and use it?

I believe this is a very important and valuable blog.

wu-sheng requested review from kezhenxu94 and Superskyyy October 25, 2022 03:27

jaychoww changed the title ~~add GreenletProfiler~~ feat: add GreenletProfiler Oct 25, 2022

jaychoww added 2 commits October 28, 2022 23:58

add GreenletProfiler

6caa2cd

e2e test cases

98f2d81

jaychoww force-pushed the feat_greenlet branch from 29d0adb to 98f2d81 Compare October 28, 2022 16:00

jaychoww marked this pull request as ready for review October 28, 2022 16:00

delete flask_consumer.py

7d5c10a

sonatype-lift bot reviewed Oct 28, 2022

View reviewed changes

Licensed

7f431b4

jaychoww marked this pull request as draft October 28, 2022 16:33

handle BlockingSwitchOutError

cd678a7

jaychoww marked this pull request as ready for review October 29, 2022 10:14

sonatype-lift bot reviewed Oct 29, 2022

View reviewed changes

tests/e2e/case/profiling/provider/provider.py Outdated Show resolved Hide resolved

jaychoww added 4 commits October 29, 2022 19:07

delete debug logs

318e280

lint

f05f447

lint

3dfc1c0

run profiling e2e cases in CI

c7eac8f

Superskyyy requested changes Nov 1, 2022

View reviewed changes

.github/workflows/CI.yaml Outdated Show resolved Hide resolved

jaychoww added 2 commits November 2, 2022 10:38

check greenlet if version < 2.0.0

694830d

typo

6ffd51c

wu-sheng added the feature New feature label Nov 2, 2022

Superskyyy self-requested a review November 4, 2022 17:12

Superskyyy previously approved these changes Nov 4, 2022

View reviewed changes

Superskyyy added this to the 1.0.0 milestone Nov 4, 2022

kezhenxu94 previously approved these changes Nov 5, 2022

View reviewed changes

skywalking/profile/profile_context.py Outdated Show resolved Hide resolved

compare versions by packaging.version

58ba602

jaychoww dismissed stale reviews from kezhenxu94 and Superskyyy via 58ba602 November 5, 2022 03:22

kezhenxu94 approved these changes Nov 5, 2022

View reviewed changes

wu-sheng reviewed Nov 5, 2022

View reviewed changes

kezhenxu94 merged commit 11a74be into apache:master Nov 5, 2022

jaychoww deleted the feat_greenlet branch November 9, 2022 06:36

jaychoww added a commit to jaychoww/skywalking-python that referenced this pull request Oct 24, 2023

feat: add GreenletProfiler (apache#246)

ab527d1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add GreenletProfiler #246

feat: add GreenletProfiler #246

jaychoww commented Oct 25, 2022

jaychoww commented Oct 25, 2022 •

edited

Loading

wu-sheng commented Oct 25, 2022

wu-sheng commented Oct 25, 2022

jaychoww commented Oct 25, 2022

sonatype-lift bot Oct 28, 2022

sonatype-lift bot Oct 28, 2022

jaychoww commented Oct 28, 2022 •

edited

Loading

Superskyyy commented Nov 2, 2022

Superskyyy commented Nov 4, 2022 •

edited

Loading

Superskyyy commented Nov 4, 2022 •

edited

Loading

jaychoww commented Nov 5, 2022

kezhenxu94 left a comment

wu-sheng left a comment

Command	Usage
`@sonatype-lift ignore`	Leave out the above finding from this PR
`@sonatype-lift ignoreall`	Leave out all the existing findings from this PR
`@sonatype-lift exclude <file\|issue\|path\|tool>`	Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file

feat: add GreenletProfiler #246

feat: add GreenletProfiler #246

Conversation

jaychoww commented Oct 25, 2022

jaychoww commented Oct 25, 2022 • edited Loading

wu-sheng commented Oct 25, 2022

wu-sheng commented Oct 25, 2022

jaychoww commented Oct 25, 2022

sonatype-lift bot Oct 28, 2022

Choose a reason for hiding this comment

sonatype-lift bot Oct 28, 2022

Choose a reason for hiding this comment

jaychoww commented Oct 28, 2022 • edited Loading

Superskyyy commented Nov 2, 2022

Superskyyy commented Nov 4, 2022 • edited Loading

Superskyyy commented Nov 4, 2022 • edited Loading

jaychoww commented Nov 5, 2022

kezhenxu94 left a comment

Choose a reason for hiding this comment

wu-sheng left a comment

Choose a reason for hiding this comment

jaychoww commented Oct 25, 2022 •

edited

Loading

jaychoww commented Oct 28, 2022 •

edited

Loading

Superskyyy commented Nov 4, 2022 •

edited

Loading

Superskyyy commented Nov 4, 2022 •

edited

Loading