Skip to content

Commit

Permalink
Close monitoring processes at shutdown, freeing some fds
Browse files Browse the repository at this point in the history
Before this PR, pytest parsl/tests/test_monitoring/ leaves 1093 fds open
(which is too much for the test suite to complete in some default
configurations of max fds 1024)

After this PR, that test run leaves 895 fds open.
  • Loading branch information
benclifford committed May 2, 2024
1 parent 6e48507 commit 4481985
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions parsl/monitoring/monitoring.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ def close(self) -> None:
self.router_exit_event.set()
logger.info("Waiting for router to terminate")
self.router_proc.join()
self.router_proc.close()
logger.debug("Finished waiting for router termination")
if len(exception_msgs) == 0:
logger.debug("Sending STOP to DBM")
Expand All @@ -252,13 +253,15 @@ def close(self) -> None:
logger.debug("Not sending STOP to DBM, because there were DBM exceptions")
logger.debug("Waiting for DB termination")
self.dbm_proc.join()
self.dbm_proc.close()
logger.debug("Finished waiting for DBM termination")

# should this be message based? it probably doesn't need to be if
# we believe we've received all messages
logger.info("Terminating filesystem radio receiver process")
self.filesystem_proc.terminate()
self.filesystem_proc.join()
self.filesystem_proc.close()

logger.info("Closing monitoring multiprocessing queues")
self.exception_q.close()
Expand Down

0 comments on commit 4481985

Please sign in to comment.