Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(executor): override default resources to remove mem/disk (#91) #91

Merged
merged 1 commit into from
Mar 22, 2024

Conversation

mdonadoni
Copy link
Member

Closes #90

mdonadoni added a commit to mdonadoni/reana-workflow-engine-snakemake that referenced this pull request Mar 21, 2024
Copy link
Member

@tiborsimko tiborsimko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works nicely 👍

$ rcg logs -w root6-roofit-snakemake-kubernetes.1 | grep resources:
    resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=<TBD>, kubernetes_memory_limit=256Mi
    resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=<TBD>, kubernetes_memory_limit=256Mi
    resources: mem_mb=1000, mem_mib=954, disk_mb=1000, disk_mib=954, tmpdir=/tmp

$ rcg logs -w root6-roofit-snakemake-kubernetes.2 | grep resources:
    resources: tmpdir=<TBD>, kubernetes_memory_limit=256Mi
    resources: tmpdir=<TBD>, kubernetes_memory_limit=256Mi
    resources: tmpdir=/tmp

Left a minor note on the explaining comment.

Since this is a well-isolated fix, perhaps it should go to maint-0.9 as well...

config=workflow_parameters,
workdir=workflow_workspace,
keep_logger=True,
# Since Snakemake v7.3.0, if default resources are not specified when in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A suggestion for the comment:

        # Since Snakemake v7.3.0, the workflow logs include Snakemake-percieved native
        # resource information on memory and storage (`mem_mb`, `disk_mb`) for each
        # Snakemake rule run on cloud. However, REANA overrides these when running 
        # user jobs, so we should hide these in order not to present any misleading 
        # information to users. For this reason, the default resources are overridden 
        # here with the only the "bare" ones (`tmpdir`).

@mdonadoni mdonadoni changed the base branch from master to maint-0.9 March 22, 2024 09:03
mdonadoni added a commit to mdonadoni/reana-workflow-engine-snakemake that referenced this pull request Mar 22, 2024
Copy link

codecov bot commented Mar 22, 2024

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 3.78%. Comparing base (f63e5cb) to head (572a83f).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##           maint-0.9     #91      +/-   ##
============================================
- Coverage       3.82%   3.78%   -0.05%     
============================================
  Files              6       6              
  Lines            183     185       +2     
============================================
  Hits               7       7              
- Misses           176     178       +2     
Files Coverage Δ
reana_workflow_engine_snakemake/executor.py 0.00% <0.00%> (ø)

@mdonadoni mdonadoni merged commit 572a83f into reanahub:maint-0.9 Mar 22, 2024
13 checks passed
mdonadoni added a commit to mdonadoni/reana-workflow-engine-snakemake that referenced this pull request Mar 22, 2024
* maint-0.9:
  fix(executor): override default resources to remove mem/disk (reanahub#91)
mdonadoni added a commit to mdonadoni/reana-workflow-engine-snakemake that referenced this pull request Mar 22, 2024
* fix(executor): override default resources to remove mem/disk (reanahub#91)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

logs: misleading memory and disk limit
2 participants