Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow jobs that are immediately ready-to-run to skip the promote cycle. #22

Closed
vsivsi opened this issue Jul 30, 2014 · 12 comments
Closed

Comments

@vsivsi
Copy link
Owner

vsivsi commented Jul 30, 2014

See: #15 (comment)

@vsivsi
Copy link
Owner Author

vsivsi commented Jul 30, 2014

One slightly tricky thing here is that some of the methods impacted can work on multiple jobs simultaneously. So some jobs may be ready and others not...

  • .save() -- Only one job at a time
  • .rerun() -- Only one job at a time
  • .resume() -- Multiple jobs
  • .restart() -- Multiple jobs

@mitar
Copy link
Collaborator

mitar commented Jul 30, 2014

Maybe you could add .save().promote()? But it would be better that this is called automatically.

@mitar
Copy link
Collaborator

mitar commented Jul 30, 2014

Hm, but pool() is quite efficient, not sure if this is a problem on a busy server? Only one query more?

@vsivsi
Copy link
Owner Author

vsivsi commented Jul 30, 2014

Try out the instant_ready branch and let me know what you think. Should be completely automatic.

@mitar
Copy link
Collaborator

mitar commented Jul 30, 2014

OK, but it is doing one more query? I don't think this is a too big issue.

@vsivsi
Copy link
Owner Author

vsivsi commented Jul 30, 2014

Yes, it's doing another update, but restricted to the id(s) involved in the invoking method, which should make it much faster than the full query used when promote polls.

@mitar
Copy link
Collaborator

mitar commented Jul 30, 2014

I like it. Thanks!

@mitar
Copy link
Collaborator

mitar commented Jul 31, 2014

This is quick now! Especially with my observe-based worker. :-)

Now I just have to see what happens when there are dependencies between documents. Do I have to wait for promote to discover that, or does it set ready state automatically after done to all dependent jobs.

@vsivsi
Copy link
Owner Author

vsivsi commented Jul 31, 2014

Dependencies are resolved by job.done(), which doesn't cause the ready state of any affected jobs to be evaluated. So, you are correct, they will wait until the next promote cycle.

@mitar
Copy link
Collaborator

mitar commented Jul 31, 2014

But this would be easy to add, no? Using the same technique, calling pool on all dependent jobs? Maybe this could be an option to done(), to make it do this two additional queries (one to find dependent jobs, one to update them)?

@vsivsi
Copy link
Owner Author

vsivsi commented Feb 8, 2015

This has been committed on the dev branch

@vsivsi
Copy link
Owner Author

vsivsi commented Mar 24, 2015

merged into master

@vsivsi vsivsi closed this as completed Mar 24, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants