Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remote input file download slowdown submission rate #1206

Closed
pditommaso opened this issue Jun 27, 2019 · 1 comment
Closed

Remote input file download slowdown submission rate #1206

pditommaso opened this issue Jun 27, 2019 · 1 comment
Labels
bug
Milestone
v19.07.0

Comments

@pditommaso
Copy link
Member

The remote input file caching feature introduced by #686 slows down the tasks submission rate because cause the task submission to wait for the remote file download completion.
pditommaso added a commit that referenced this issue Jul 9, 2019
@pditommaso
Fix Remote input file download slowdown submission rate #1206
b710e21
pditommaso added a commit that referenced this issue Jul 9, 2019
@pditommaso
Improve FileTransfer await termination log #1206
cb968da 
This implements a better logging when nextflow stops to await
the completion of ongoing file transters.

It prints a message in the console every 10 minutes reporting
the number of uploads to complete and a message in the log file
every minute with the same information.
@pditommaso pditommaso added this to the v19.07.0 milestone Jul 9, 2019
@JohnHadish JohnHadish mentioned this issue Aug 29, 2019
Closed
@JohnHadish
Copy link

I am a big fan of this fix, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
v19.07.0
Development

No branches or pull requests
3 participants
@pditommaso @bentsherman @JohnHadish