-
-
Notifications
You must be signed in to change notification settings - Fork 30.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove the usage of the C stack in Python to Python calls #89419
Comments
Removing the usage of the C stack in Python-to-Python calls will allow future optimizations in the eval loop to take place and can yield some speed ups given that we will be removing C function calls and preambles by inlining the callee in the same eval loop as the caller. |
AFAIK Mark Shannon proposed this idea, but it was rejected. |
What was rejected was https://www.python.org/dev/peps/pep-0651/ which included this idea but had a lot more stuff in it. In particular, it was rejected because it gave semantics to overflow exceptions (two exceptions were proposed), new APIs and it had lack consistent guarantees for different platforms, among other considerations. This version is just for optimization purposes with no changes in semantics. |
Hey guys, you know that you are about to implement the core idea of Stackless Python, right? :-D |
I've trying to do this since about 2011 :) |
I fully support implementing the core idea of Stackless Python :) I spent a whole EuroPython a couple of years back discussing the idea with (apparently) everyone except Mark. Though I wouldn't like to lose the ability to extract the Python stack by inspecting native memory alone. That is very useful for debugging, particularly when you've only got a crash dump. So provided all the code objects are only an indirection or two away from a local (or better yet, parameter) value, it should be fine. |
Don't worry about it, I am personally making sure that keeps being possible. it will need some changes in the tools, but not any more that any other changes between minor versions of the interpreter |
FYI., in Stackless Python I built a chain of frames by a double linked list. This was the replacement for the current frame/stack mix. Note that it is a rather easy step to add the capability to squirrel the whole current chain away and replace it by another one. Add some header to such a chain, and you can call it "Tasklet". |
PR 28488 has no NEWS entry, or What's New entry. However, adding multiple entries will be confusing, so that's best left until all calls to Python functions and method don't use the C stack. |
FWIW, getting all function to avoid the C stack will most probably take a long time, if it happens at all. Especially functions which might call into Python multiple times must be re-designed heavily, turned into multiple pieces to be in tail position. Been there, long ago... :) |
We are starting to optimize first the easy cases of CALL_FUNCTION and move slowly to add more and more calls into it. This approach has the advantage that allow us to take it in small increments. |
Very much appreciated approach. |
The goal is reduced stack depth, not reframing the entire call model around not having a C stack. We can't even reasonably rewrite getattr() without supporting callbacks from C into Python, so further generalisation is very unlikely. But if you inspect the native stack of most Python programs, you'll see that it's mostly taken up with calls within Python code. Compressing all of those is a significant advantage, akin to inlining the Python code at compile time, even if it doesn't see "through" native calls. |
Unfortunately, seems that #28937 has broken the AMD64 FreeBSD Shared 3.x buildbot: https://buildbot.python.org/all/#/builders/483/builds/1003/steps/5/logs/stdio The buildbot was green until we merged this |
https://bugs.python.org/issue45829 is the related issue for special methods |
… no frame is available ``` Unable to read information on python frame Python Exception <class 'AttributeError'>: 'NoneType' object has no attribute 'co_name' ``` Regression in commit b4903af. While refactoring the code into a while loop, the previous early return when no frame exists went missing. We have just printed a message that we cannot get information about this, so the frame will be None, and we cannot attempt to use it. Discovered on python 3.11, in python 3.12a2 this should error out with `.is_shim()` instead of `co_name`. (cherry picked from commit 8586949)
… no frame is available (pythonGH-100611) ``` Unable to read information on python frame Python Exception <class 'AttributeError'>: 'NoneType' object has no attribute 'co_name' ``` Regression in commit b4903af. While refactoring the code into a while loop, the previous early return when no frame exists went missing. We have just printed a message that we cannot get information about this, so the frame will be None, and we cannot attempt to use it. Discovered on python 3.11, in python 3.12a2 this should error out with `.is_shim()` instead of `co_name`. (cherry picked from commit 8586949)
…n no frame is available (GH-100611) (#100738) gh-89419: gdb: fix bug causing AttributeError in py-locals when no frame is available (GH-100611) ``` Unable to read information on python frame Python Exception <class 'AttributeError'>: 'NoneType' object has no attribute 'co_name' ``` Regression in commit b4903af. While refactoring the code into a while loop, the previous early return when no frame exists went missing. We have just printed a message that we cannot get information about this, so the frame will be None, and we cannot attempt to use it. Discovered on python 3.11, in python 3.12a2 this should error out with `.is_shim()` instead of `co_name`. (cherry picked from commit 8586949)
* Rename Lib/test/crashers/ to Lib/test/test_crashers/. * Move Lib/test/test_crashers.py to Lib/test/test_crashers/__init__.py. * test_crashers is no longer skipped and makes sure that scripts do crash, and no simply fail with a non-zero exit code. * Update bogus_code_obj.py to use CodeType.replace(). * Remove Lib/test/crashers/ scripts which no longer crash: * recursive_call.py: fixed by pythongh-89419 * mutation_inside_cyclegc.py: fixed by pythongh-97922 * trace_at_recursion_limit.py: fixed by Python 3.7
* Rename Lib/test/crashers/ to Lib/test/test_crashers/. * Move Lib/test/test_crashers.py to Lib/test/test_crashers/__init__.py. * test_crashers is no longer skipped and makes sure that scripts do crash, and no simply fail with a non-zero exit code. * Update bogus_code_obj.py to use CodeType.replace(). * Scripts crashing Python now uses SuppressCrashReport of test.support to not create coredump files. * Remove Lib/test/crashers/ scripts which no longer crash: * recursive_call.py: fixed by pythongh-89419 * mutation_inside_cyclegc.py: fixed by pythongh-97922 * trace_at_recursion_limit.py: fixed by Python 3.7
* Rename Lib/test/crashers/ to Lib/test/test_crashers/. * Move Lib/test/test_crashers.py to Lib/test/test_crashers/__init__.py. * test_crashers is no longer skipped and makes sure that scripts do crash, and no simply fail with a non-zero exit code. * Update bogus_code_obj.py to use CodeType.replace(). * Scripts crashing Python now uses SuppressCrashReport of test.support to not create coredump files. * Remove Lib/test/crashers/ scripts which no longer crash: * recursive_call.py: fixed by pythongh-89419 * mutation_inside_cyclegc.py: fixed by pythongh-97922 * trace_at_recursion_limit.py: fixed by Python 3.7
InterpreterFrame
#30372Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
The text was updated successfully, but these errors were encountered: