-
Notifications
You must be signed in to change notification settings - Fork 866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Jobs are not in order or not shown in order when displayed in the UI #33
Comments
haha yeah i've definitely noticed that too. but i was testing something else so i didn't look into it much yet |
hmm it seems like they are ordered correct, however without a refresh they seem to be tacked on the bottom regardless of priority, kinda funky, looking into it |
oh nvm it's because the dom elements are not sorted so existing ones stay where they are |
verified that they are processed in proper order as well |
As of version 0.4.1, they are still not processed in proper order. A queue is intuitively expected to be FIFO, but redis sorted set is sorting the elements numerically only for the score. Kue implementation is to use the priority number for redis score. This means that a job with the same priority is sorted based on the id as STRING instead of NUMBER. A job with id 10 will be processed before job with id 2. I think this is unintuitive and a violation of the queue principle. This bug is clearly demonstrated in this screencast: http://nodetuts.com/tutorials/27-kue-jobs.html#video |
I have the same problem... any ideas of how can this be fixed? |
First, Jobs are ordered by their priority not job ids. |
Yep, I got it. But I noticed the commits above provides sorting function by id, it works well as it should be therefore when the UI web scrolling down, loading more data(by increasing ends range) the job list did't sorted correctly(the json api's output is always ordered by id) that's what I mentioned. |
That sort function has been merged long ago...
Anyone can help providing a patch? |
So this issue should be reopen? Because the issue just had been solved partially (JSON API). |
This should be handled in client in an efficient way, any patch is welcome. |
I am still seeing an issue of jobs processing in the incorrect order. I want to process earlier jobs first. I understand that priority has precedence over id but i need to process an arbitrary amount of jobs where the only sorting criteria is earlier jobs get processed before later jobs. If I set all jobs to have the same priority, then the order jobs get processed is Is there a way I can force FIFO behavior on the queue so that jobs get processed with ids Perhaps the kue code needs to be changed to something like this answer http://stackoverflow.com/a/13820749 to maintain the proper sorting in redis. Right now when I look at the list of inactive
|
Yes, I was aware of this. The source is lexicographic ordering of set id's in redis. That link provides an implementation which is simple but ugly, let me think more on this, Thank you for the link. |
One option is to zero pad the to the front of the job id when saving to the sorted set in redis, and then call One downside is there an upper bound on how many jobs can be stored that is controlled by the amount of zeros prepended to the front var id = 1
var redisJobID = 000000001
// add job to redis inactive sorted set
...
/// Then change the zpop to call parseInt on the reply from redis
self.zpop(client.getKey('jobs:' + self.type + ':inactive'), function (err, id) {
if (err || !id) {
self.job = null;
return fn(err /*|| "No job to pop!"*/);
}
id = parseInt(id, 10)
Job.get(id, fn); |
I put together a sample implementation here https://github.com/nisaacson/kue/tree/padded-job-id |
@nisaacson : Instead of appending zeroes, we can avoid the upper limit by pre-pending to the string a byte such as Additionally, if we require a very recent version of redis, we can specify a substitute string for lexicographic ordering on zadd, that will be used on zrangebylex: http://redis.io/commands/zrangebylex |
getting sorting to work right on the UI would be awesome 👍 |
the sorting issue is not related to UI and is builtin in regard with how Redis sorting works. Please read linked issues above in detail... |
The fact that this bug is still open means kue does not actually behave as a priority queue? This sounds weird, however, since kue is avertised as a "priority job queue"... |
The problem isn't in the job queue or the priority of those jobs, but in the display of the jobs in a web based UI. Often the jobs are not displayed in a sorted (numeric) order, which has been an annoyance for using kue at least for me. |
this is sure a bug within kue, the source is how redis behaves on Sorted Set key collations, However this wont affect the prioritization of jobs, but sorting them. |
Thanks for the clarification. El jue, mar 5, 2015 21:11, Behrad notifications@github.com escribió:
|
and this happens just for job ids when most significant digit is moved forward 1 |
this is moved to 1.0.0 |
Fixed in 0.10.3 |
See what I mean here: http://twitpic.com/5mpuqo
Looks like it's being shown in alpha order instead of numerical.
The text was updated successfully, but these errors were encountered: