Skip to content

Conversation

@PythonFZ
Copy link
Contributor

@PythonFZ PythonFZ commented Nov 23, 2022

Currently dvc exp run --run-all --jobs <n> will only start a single celery worker.
This PR would re-enable spawing multiple workers.

I did test this on my machine but I'm not familiar with the dvc test suit so I wasn't able to add a proper unit test for this feature.

Note: the worker id changes from a default 1 to a default 0 with this PR. If this is important the range can be easily adjusted.

@codecov
Copy link

codecov bot commented Nov 23, 2022

Codecov Report

Base: 93.52% // Head: 93.52% // No change to project coverage πŸ‘

Coverage data is based on head (bce0537) compared to base (6fc0a6f).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8608   +/-   ##
=======================================
  Coverage   93.52%   93.52%           
=======================================
  Files         457      457           
  Lines       36128    36128           
  Branches     5224     5224           
=======================================
  Hits        33790    33790           
  Misses       1837     1837           
  Partials      501      501           
Impacted Files Coverage Ξ”
tests/unit/command/test_queue.py 100.00% <ΓΈ> (ΓΈ)
dvc/repo/experiments/__init__.py 87.00% <100.00%> (ΓΈ)
dvc/repo/experiments/queue/celery.py 85.20% <100.00%> (ΓΈ)
tests/unit/repo/experiments/conftest.py 88.67% <100.00%> (ΓΈ)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

β˜” View full report at Codecov.
πŸ“’ Do you have feedback about the report comment? Let us know in this issue.

@daavoo daavoo requested a review from pmrowla November 23, 2022 13:52
@daavoo daavoo added A: experiments Related to dvc exp A: task-queue labels Nov 23, 2022
@pmrowla pmrowla changed the title re-enable --jobs concurrency exp run --run-all: re-enable --jobs concurrency Nov 24, 2022
@PythonFZ PythonFZ requested a review from pmrowla December 16, 2022 16:27
@pmrowla pmrowla merged commit 28848c5 into treeverse:main Dec 26, 2022
@pmrowla
Copy link
Contributor

pmrowla commented Dec 26, 2022

Thanks for the fix @PythonFZ!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A: experiments Related to dvc exp

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants