Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to immediately shutdown executor #676

Merged
merged 2 commits into from
Feb 26, 2014

Conversation

jamesls
Copy link
Member

@jamesls jamesls commented Feb 25, 2014

Switch the task queue to a priority queue and allow
ShutdownThreadRequest (which originally were just the QUEUE_END_SENTINEL
object) to have a more important priority in the case that a user
has hit Ctrl+C.

This is also needed to fully fix: #657

Previously, not all the integ tests were passing. Now I can all unit/integ tests passing with this change.

Switch the task queue to a priority queue and allow
ShutdownThreadRequest (which originally were just the QUEUE_END_SENTINEL
object) to have a more important priority in the case that a user
has hit Ctrl+C.
@danielgtaylor
Copy link
Contributor

LGTM 👍

@jamesls
Copy link
Member Author

jamesls commented Feb 26, 2014

@danielgtaylor Sorry about this, but there's going to be one more change coming shortly.

The PriorityQueue is backed by a heap which does not provide a stable sort order, whereas we need FIFO semantics for tasks of the same priority. I have a fix I'm testing but I want to be absolutely sure this time before I have you review it again.

This is needed to provided FIFO semantics to the task queue
for objects of the same priority.

We only need to be able to preempt workers with high priority
items.  Otherwise everything should be in FIFO order.
@jamesls
Copy link
Member Author

jamesls commented Feb 26, 2014

I've updated this to use a stable priority queue. Would you mind taking another look?

@danielgtaylor
Copy link
Contributor

The new stable priority queue looks good, as do the tests. 🚢-it!

@jamesls jamesls merged commit 63e9e46 into aws:develop Feb 26, 2014
@jamesls jamesls deleted the immediate-s3-exit branch June 23, 2014 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sync and cp hangs with a large amount of files
2 participants