Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Execute the garbage collector only on the eval breaker #97922

Closed
pablogsal opened this issue Oct 5, 2022 · 1 comment
Closed

Execute the garbage collector only on the eval breaker #97922

pablogsal opened this issue Oct 5, 2022 · 1 comment

Comments

@pablogsal
Copy link
Member

Currently, the GC can be executed on every object allocation. This has been historically the source of many problems because it can trigger a GC run in points where the VM is in an inconsistent state. This includes critical points of the eval loop but also during complex object creation since the GC can run while creating sub-elements of the final result meanwhile the object is not fully initialized.

To improve the situation, we can schedule a GC run on object allocation but the GC will only run then on the eval breaker in a similar fashion we currently use to run signal handlers, do GIL switch and run pending callbacks.
@bedevere-bot bedevere-bot mentioned this issue Oct 5, 2022
Merged
@pablogsal
Copy link
Member Author

One small consideration:

We can opportunistically check if the GC is scheduled to run and run it if we have a request in PyErr_CheckSignals. This can benefit native code that needs to call PyErr_CheckSignals if is going to run for some time without executing Python code to ensure signals are handled. Checking for the GC here allows long-running native code to clean cycles created using the C-API even if it doesn't run the evaluation loop

pablogsal added a commit that referenced this issue Oct 8, 2022
@pablogsal
gh-97922: Run the GC only on eval breaker (#97920)
83eb827
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
@carljm
Merge branch 'main' into typewatch
d67d6fd 
* main:
  pythongh-68686: Retire eptag ptag scripts (python#98064)
  pythongh-97922: Run the GC only on eval breaker (python#97920)
  GitHub Workflows security hardening (python#96492)
  Add `@ezio-melotti` as codeowner for `.github/`. (python#98079)
  pythongh-97913 Docs: Add walrus operator to the index (python#97921)
  [doc] Fix broken links to C extensions accelerating stdlib modules (python#96914)
  pythongh-97822: Fix http.server documentation reference to test() function (python#98027)
  pythongh-91052: Add PyDict_Unwatch for unwatching a dictionary (python#98055)
  pythonGH-98023: Change default child watcher to PidfdChildWatcher on supported systems (python#98024)
  pythonGH-94182: Run the PidfdChildWatcher on the running loop (python#94184)
carljm added a commit to carljm/cpython that referenced this issue Oct 9, 2022
@carljm
Merge branch 'main' into urllibnonet
132c32e 
* main: (5519 commits)
  Minor edits to the Descriptor HowTo Guide (pythonGH-24901)
  Fix link to Lifecycle of a Pull Request in CONTRIBUTING (python#98102)
  pythonGH-94597: deprecate `SafeChildWatcher`, `FastChildWatcher` and `MultiLoopChildWatcher` child watchers  (python#98089)
  Auto-cancel old builds when new commit pushed to branch (python#98009)
  pythongh-95011: Migrate syslog module to Argument Clinic (pythonGH-95012)
  pythongh-68686: Retire eptag ptag scripts (python#98064)
  pythongh-97922: Run the GC only on eval breaker (python#97920)
  GitHub Workflows security hardening (python#96492)
  Add `@ezio-melotti` as codeowner for `.github/`. (python#98079)
  pythongh-97913 Docs: Add walrus operator to the index (python#97921)
  [doc] Fix broken links to C extensions accelerating stdlib modules (python#96914)
  pythongh-97822: Fix http.server documentation reference to test() function (python#98027)
  pythongh-91052: Add PyDict_Unwatch for unwatching a dictionary (python#98055)
  pythonGH-98023: Change default child watcher to PidfdChildWatcher on supported systems (python#98024)
  pythonGH-94182: Run the PidfdChildWatcher on the running loop (python#94184)
  pythongh-92886: make test_ast pass with -O (assertions off) (pythonGH-98058)
  pythongh-92886: make test_coroutines pass with -O (assertions off) (pythonGH-98060)
  pythongh-57179: Add note on symlinks for os.walk (python#94799)
  pythongh-94808: Fix regex on exotic platforms (python#98036)
  pythongh-90085: Remove vestigial -t and -c timeit options (python#94941)
  ...
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
@pablogsal @mpage
pythongh-97922: Run the GC only on eval breaker (python#97920)
65d36ba
@markshannon markshannon mentioned this issue Oct 14, 2022
Open
@vstinner vstinner mentioned this issue Nov 9, 2022
Merged
@vstinner vstinner mentioned this issue Nov 24, 2022
Closed
@Adriani90 Adriani90 mentioned this issue Apr 21, 2023
Closed
vstinner added a commit to vstinner/cpython that referenced this issue Aug 22, 2023
@vstinner
pythongh-108297: Update test_crashers
e4dced8 
* Rename Lib/test/crashers/ to Lib/test/test_crashers/.
* Move  Lib/test/test_crashers.py to
  Lib/test/test_crashers/__init__.py.
* test_crashers is no longer skipped and makes sure that scripts do
  crash, and no simply fail with a non-zero exit code.
* Update bogus_code_obj.py to use CodeType.replace().
* Remove Lib/test/crashers/ scripts which no longer crash:

  * recursive_call.py: fixed by pythongh-89419
  * mutation_inside_cyclegc.py: fixed by pythongh-97922
  * trace_at_recursion_limit.py: fixed by Python 3.7
vstinner added a commit to vstinner/cpython that referenced this issue Aug 22, 2023
@vstinner
pythongh-108297: Update test_crashers
00b9633 
* Rename Lib/test/crashers/ to Lib/test/test_crashers/.
* Move  Lib/test/test_crashers.py to
  Lib/test/test_crashers/__init__.py.
* test_crashers is no longer skipped and makes sure that scripts do
  crash, and no simply fail with a non-zero exit code.
* Update bogus_code_obj.py to use CodeType.replace().
* Scripts crashing Python now uses SuppressCrashReport of
  test.support to not create coredump files.
* Remove Lib/test/crashers/ scripts which no longer crash:

  * recursive_call.py: fixed by pythongh-89419
  * mutation_inside_cyclegc.py: fixed by pythongh-97922
  * trace_at_recursion_limit.py: fixed by Python 3.7
@vstinner vstinner mentioned this issue Aug 22, 2023
Closed
vstinner added a commit to vstinner/cpython that referenced this issue Aug 22, 2023
@vstinner
pythongh-108297: Update test_crashers
895f2fa 
* Rename Lib/test/crashers/ to Lib/test/test_crashers/.
* Move  Lib/test/test_crashers.py to
  Lib/test/test_crashers/__init__.py.
* test_crashers is no longer skipped and makes sure that scripts do
  crash, and no simply fail with a non-zero exit code.
* Update bogus_code_obj.py to use CodeType.replace().
* Scripts crashing Python now uses SuppressCrashReport of
  test.support to not create coredump files.
* Remove Lib/test/crashers/ scripts which no longer crash:

  * recursive_call.py: fixed by pythongh-89419
  * mutation_inside_cyclegc.py: fixed by pythongh-97922
  * trace_at_recursion_limit.py: fixed by Python 3.7
@EliahKagan EliahKagan mentioned this issue Nov 28, 2023
Merged
@vstinner vstinner mentioned this issue Dec 7, 2023
Open
@yilei yilei mentioned this issue Dec 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@pablogsal