New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sys.settrace dramatic slowdown in 3.12 #107674
Comments
This issue can be seen starting from the commit 411b169 with usual numpy-1.25.2. |
cc. @markshannon |
…events (pythonGH-107780) (cherry picked from commit 37d8b90) Co-authored-by: Mark Shannon <mark@hotpy.org>
@markshannon Is there any news about this? |
@iritkatriel perhaps you have some idea? |
Did #107780 help? |
@gaogaotiantian Thanks. I tried #114986, and it didn't run faster. I used the https://github.com/Quansight-Labs/ndindex repo test suite: I wish I had better news. |
Oh, did you test it with an empty trace function? |
I will do more tests, but the ultimate goal is to perform better in real-world scenarios. Here is more data from a subset of the test suite (
|
Thank you for the test. Yes the PR only improves the performance of the actual trace function - which is normally the major cost for tracing tools. Good to know it helps with the more real-life benchmark. Unfortunately, due to the change of the tracing mechanism, the overhead for |
Now that the eval_breaker is part of the thread state, here's an alternative approach that might work.
If the code and global versions differ do the following:
I think this is correct, but I've not drawn up a full state diagram. |
#114986 caused refleaks tests ( |
…nGH-114986)" This reverts commit 0a61e23.
….settrace` (pythonGH-114986)" (pythonGH-116178) Revert "pythongh-107674: Improve performance of `sys.settrace` (pythonGH-114986)" This reverts commit 0a61e23.
….settrace` (pythonGH-114986)" (pythonGH-116178) Revert "pythongh-107674: Improve performance of `sys.settrace` (pythonGH-114986)" This reverts commit 0a61e23.
Hi. I wanted to post this in nedbat/coveragepy#1665 but it seems that investigation moved here. I hope my code example will help to investigate and not clutter the Issue. I also have been chasing "tests run 2 times slower" dragon. I used
This one is from the project. Let's call it
Files and commands to reproduce
# fut.py
import jieba
def a():
pass #
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --allow-unsafe --output-file=requirements.txt requirements.in
#
coverage[toml]==7.4.1
# via pytest-cov
iniconfig==2.0.0
# via pytest
jieba==0.42.1
# via -r requirements.in
packaging==23.2
# via pytest
pluggy==1.4.0
# via pytest
pytest==8.1.1
# via
# -r requirements.in
# pytest-cov
pytest-cov==4.1.0
# via -r requirements.in # test/test_fut.py
import fut
def test_stuff():
assert True System info:
When run trough
|
Ah, this is interesting. The problem is that the file has a long const dict that expands into 30k+ lines. Each line will execute to produce part of the dict, and trigger a callback into the trace function, which will slow down the program significantly. This is not a bug, and almost expected - it's just the unfortunate code that makes it weird. |
….settrace` (pythonGH-114986)" (pythonGH-116178) Revert "pythongh-107674: Improve performance of `sys.settrace` (pythonGH-114986)" This reverts commit 0a61e23.
I believe this performance regression is causing debugging to become practically unusable in 3.12+ when using certain dependencies. This repro takes two and a half minutes to run on a Macbook pro, due to the # Run with python -m pdb repro.py
# Type n to execute the import statement with an active breakpoint
from phonenumbers import geocoder, parse
print(geocoder.description_for_number(parse('+13215555555'), 'en')) Perhaps the library can be updated to use the fast C-based JSON scanner for performance, but that's a significant change to make to work around a language performance regression. |
Just to confirm, what command did you use after |
Good point. I typed |
Yes, |
Performance improvements in any future version would be great. That said, it is conceivable to me that generating very large Python files will become a library anti-pattern due to this. |
I came up with a patch to fix the long dict issue. I'm not sure if you are interested in trying the patch in #118127. If you are, let me know the result. |
* Check tracing in RESUME_CHECK * Only change to RESUME_CHECK if not tracing
Use PEP669's sys.monitoring for faster coverage analysis on GitHub Actions, if available. This also fixes a significant regression in coverage analysis on Python 3.12. Related issue: python/cpython#107674 .
* Check tracing in RESUME_CHECK * Only change to RESUME_CHECK if not tracing
Use PEP669's sys.monitoring for faster coverage analysis on GitHub Actions, if available. This also fixes a significant regression in coverage analysis on Python 3.12. Related issue: python/cpython#107674 .
Python 3.12 has a significant regression in coverage analysis. Related issue: python/cpython#107674 . This patch replaces coverage with a custom framework built using `sys.monitoring`, which is only available as of Python 3.12.
Bug report
Checklist
A clear and concise description of the bug
A bug report in coverage (nedbat/coveragepy#1665) is reduced here to a pure do-nothing trace function for sys.settrace.
Reproduction:
I don't know if the unusual numpy build has something to do with this.
Linked PRs
sys.settrace
line events #107780sys.settrace
line events (GH-107780) #107841sys.settrace
#114986sys.settrace
#117133The text was updated successfully, but these errors were encountered: