Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-42118: Allow run-qbb to have implicit threading disabled #278

Merged
merged 5 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ repos:
# https://pre-commit.com/#top_level-default_language_version
language_version: python3.11
- repo: https://github.com/pycqa/isort
rev: 5.12.0
rev: 5.13.1
hooks:
- id: isort
name: isort (python)
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.1.6
rev: v0.1.7
hooks:
- id: ruff
1 change: 1 addition & 0 deletions doc/changes/DM-42118.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Ensure that the implicit threading options for ``run-qbb`` is used so that implicit threading can be disabled.
5 changes: 5 additions & 0 deletions python/lsst/ctrl/mpexec/cli/script/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
import logging
from types import SimpleNamespace

from lsst.utils.threads import disable_implicit_threading

from ... import CmdLineFwk, TaskFactory

_log = logging.getLogger(__name__)
Expand Down Expand Up @@ -187,6 +189,9 @@
start_method = "spawn"
_log.warning("Option --start-method=fork is unsafe and no longer supported, will use spawn instead.")

if not enable_implicit_threading:
disable_implicit_threading()

Check warning on line 193 in python/lsst/ctrl/mpexec/cli/script/run.py

View check run for this annotation

Codecov / codecov/patch

python/lsst/ctrl/mpexec/cli/script/run.py#L193

Added line #L193 was not covered by tests

args = SimpleNamespace(
pdb=pdb,
graph_fixup=graph_fixup,
Expand Down
5 changes: 5 additions & 0 deletions python/lsst/ctrl/mpexec/cli/script/run_qbb.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
import logging
from types import SimpleNamespace

from lsst.utils.threads import disable_implicit_threading

from ... import CmdLineFwk, TaskFactory

_log = logging.getLogger(__name__)
Expand Down Expand Up @@ -105,6 +107,9 @@
start_method = "spawn"
_log.warning("Option --start-method=fork is unsafe and no longer supported, will use spawn instead.")

if not enable_implicit_threading:
disable_implicit_threading()

Check warning on line 111 in python/lsst/ctrl/mpexec/cli/script/run_qbb.py

View check run for this annotation

Codecov / codecov/patch

python/lsst/ctrl/mpexec/cli/script/run_qbb.py#L111

Added line #L111 was not covered by tests

args = SimpleNamespace(
butler_config=butler_config,
qgraph=qgraph,
Expand Down
9 changes: 6 additions & 3 deletions python/lsst/ctrl/mpexec/cmdLineFwk.py
Original file line number Diff line number Diff line change
Expand Up @@ -750,6 +750,9 @@ def runPipeline(
Data Butler instance, if not defined then new instance is made
using command line options.
"""
if not args.enable_implicit_threading:
disable_implicit_threading()

# Check that output run defined on command line is consistent with
# quantum graph.
if args.output_run and graph.metadata:
Expand All @@ -768,9 +771,6 @@ def runPipeline(
else:
args.clobber_outputs = False

if not args.enable_implicit_threading:
disable_implicit_threading()

# Make butler instance. QuantumGraph should have an output run defined,
# but we ignore it here and let command line decide actual output run.
if butler is None:
Expand Down Expand Up @@ -970,6 +970,9 @@ def preExecInitQBB(self, task_factory: TaskFactory, args: SimpleNamespace) -> No
preExecInit.initialize(qgraph)

def runGraphQBB(self, task_factory: TaskFactory, args: SimpleNamespace) -> None:
if not args.enable_implicit_threading:
disable_implicit_threading()

# Load quantum graph.
nodes = args.qgraph_node_id or None
qgraph = QuantumGraph.loadUri(args.qgraph, nodes=nodes, graphID=args.qgraph_id)
Expand Down