Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set maximum number of processes per task #265

Closed
ClaraBuettner opened this issue May 28, 2021 · 0 comments · Fixed by #269
Closed

Set maximum number of processes per task #265

ClaraBuettner opened this issue May 28, 2021 · 0 comments · Fixed by #269
Assignees
@ClaraBuettner

Comments

@ClaraBuettner
Copy link
Contributor

Currently tasks can run in parallel (via the --jobs variable) but some tasks also include multiprocessing. So the actual number of used cores by egon-data is sometimes higher than the selected --jobs.

This should be avoided by setting the processes per task to one or adding another parameter (like e.g. max_process_per_task).
@ClaraBuettner ClaraBuettner self-assigned this May 28, 2021
@ClaraBuettner ClaraBuettner mentioned this issue Jun 1, 2021
Merged
ClaraBuettner added a commit that referenced this issue Jun 2, 2021
@ClaraBuettner
Merge branch 'dev' into features/#265-limit-number-of-processes-per-task
3120929
ClaraBuettner added a commit that referenced this issue Sep 8, 2021
@ClaraBuettner
Merge branch 'dev' into features/#265-limit-number-of-processes-per-task
57125e8
ClaraBuettner added a commit that referenced this issue Sep 9, 2021
@ClaraBuettner
Merge branch 'features/#265-limit-number-of-processes-per-task' into …
509cf02 
…continuous-integration/run-everything-over-the-weekend
ClaraBuettner added a commit that referenced this issue Oct 27, 2021
@ClaraBuettner
Merge branch 'features/#265-limit-number-of-parallel-processes-again'…
abeab74 
… into continuous-integration/run-everything-over-the-weekend
ClaraBuettner added a commit that referenced this issue Dec 2, 2021
@ClaraBuettner
Merge branch 'dev' into features/#265-limit-number-of-parallel-proces…
39fef8d 
…ses-again
ClaraBuettner added a commit that referenced this issue Dec 2, 2021
@ClaraBuettner
Merge pull request #470 from openego/features/#265-limit-number-of-pa…
805b104 
…rallel-processes-again

Features/#265 limit number of parallel processes again
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ClaraBuettner ClaraBuettner

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Limit number of parallel processes per task openego/eGon-data
1 participant
@ClaraBuettner