Improve Hub/Scheduler when no engines are registered

[minrk]

1. Tasks are pulled into the schedule, rather than left on the ZMQ queue, which means they enter the database.
2. queue_status will not raise NoEngines when there aren't any, instead it will still fetch the available information.

Bug fixed in db_query, where behavior did not match docstring (buffers should be excluded if no keys are specified).

closes #826 (again)

[ellisonbg]

I haven't tested this, but the changes are pretty simple. Just want to clarify the behavior:

• Clients can submit tasks before engines connect.
• Those tasks are pulled into the scheduler immediately and put into the DB.
• When engines connect, they are scheduled.
• If a queue_status is given before engines connect, useful information is returned.

These behaviors looks good.

[minrk]

Yes, that's correct. You could submit tasks with no engines before, the only change there is that they will show up in the Hub's db immediately, rather than hanging out in the upstream ZMQ buffer without notifying anyone. And queue_status with no engines will now succeed, returning a dict with exactly one key, enumerating these unassigned tasks (this key existed before, but engine validation prevented the request from happening):

In [4]: rc.queue_status(verbose=True)
Out[4]: {u'unassigned': [u'74b2645f-c84d-4fcf-84fb-e44706e269c2']}

[ellisonbg]

Great, I like this approach a lot. I would say merge away!

On Thu, Feb 9, 2012 at 12:39 PM, Min RK
reply@reply.github.com
wrote:

Yes, that's correct.  You could submit tasks with no engines before, the only change there is that they will show up in the Hub's db immediately, rather than hanging out in the upstream ZMQ buffer without notifying anyone.  And queue_status with no engines will now succeed, returning a dict with exactly one key, enumerating these unassigned tasks (this key existed before, but engine validation prevented the request from happening):

In [4]: rc.queue_status(verbose=True)
Out[4]: {u'unassigned': [u'74b2645f-c84d-4fcf-84fb-e44706e269c2']}

---

Reply to this email directly or view it on GitHub:
#1391 (comment)

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

[kaazoo]

Works for me so far, except that deletion of tasks doesn't seem to work. See #1400 for more information.