New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MRC-538: remove use of context/queuer #9
Conversation
Could go simpler, but these are fairly cute
Codecov Report
@@ Coverage Diff @@
## master #9 +/- ##
======================================
Coverage 100% 100%
======================================
Files 14 16 +2
Lines 849 955 +106
======================================
+ Hits 849 955 +106
Continue to review full report at Codecov.
|
This reduces the worker overhead by reducing the number of total roundtrips taken; we now have - BLPOP while waiting for job ($poll) - one pipeline while retrieving job (worker_run_task_start) - one pipeline while returning job (worker_run_task_cleanup)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great
@@ -0,0 +1,63 @@ | |||
## Adapted from queuer | |||
time_checker <- function(timeout) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a neat little function
This PR removes use of the
context
andqueuer
packages fromrrq
. Doing this will allow us to do two things:lapply
- like interface) in a way that fits better with the new environment creationThe bulk of the new code here was directly copied in from
context
/queuer
with simplifications to remove unused code branches. The code inexpression.R
will change when we do bulk submission, and includes a couple of options that are not (yet) exposed toenqueue
. The progress bar bits are all a bit tedious, but nice for interactive use.