Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-31507 Explicitly run garbage collection #135

Merged
merged 1 commit into from
Aug 26, 2021
Merged

DM-31507 Explicitly run garbage collection #135

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 8 additions & 1 deletion python/lsst/ctrl/mpexec/mpGraphExecutor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
# Imports of standard modules --
# -------------------------------
from enum import Enum
import gc
import logging
import multiprocessing
import pickle
Expand Down Expand Up @@ -348,7 +349,13 @@ def _executeQuantaInProcess(self, graph, butler):
count, totalCount = 0, len(graph)
for qnode in graph:
_LOG.debug("Executing %s", qnode)
self.quantumExecutor.execute(qnode.taskDef, qnode.quantum, butler)
try:
self.quantumExecutor.execute(qnode.taskDef, qnode.quantum, butler)
finally:
# sqlalchemy has some objects that can last until a garbage
# collection cycle is run, which can happen at unpredictable
# times, run a collection loop here explicitly.
gc.collect()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running GC frequently can be very expensive, I wonder if we could avoid pessimization by running it conditionally. In my old project when I did something similar I only ran it when virtual mem size increased significantly since last explicit CG. This of course needs some guessing as to what makes a significant increase, in that project threshold was something like 10MB, but it clearly depends on growth pattern.

count += 1
_LOG.info("Executed %d quanta, %d remain out of total %d quanta.",
count, totalCount - count, totalCount)
Expand Down