-
Notifications
You must be signed in to change notification settings - Fork 354
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(priority): change priority as a new state #1984
Merged
Merged
Changes from 20 commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
193b92a
perf(priority): change priority as a new state
roggervalf d0b196e
refactor(priority): reuse addJobWithPriority include
roggervalf 0f87cf8
chore: pass paused param in missing places
roggervalf d42e66c
refactor(change-priority): consider last job in wait list
roggervalf f386db6
refactor(pause): consider moving prioritized job to wait if needed
roggervalf 6b77001
chore: fix test cases
roggervalf 0f98015
chore: use correct variables in addJobWIthPriority
roggervalf 18b3c1d
refactor(priority): set marker when wait len is 0
roggervalf 499017f
chore: remove not needed check for last job
roggervalf 933dc97
docs: add better description for addPriorityMarkerIfNeeded
roggervalf 5c3db07
chore: update python pause script
roggervalf e36049c
chore: delete extra param
roggervalf be80d90
refactor: use timestamp for fifo order in priority zset
roggervalf b28fa1b
refactor(priority): consider fifo order with lexicographical order
roggervalf fb73c30
chore: restore python changelog
roggervalf 220e0a4
chore(python): fix scripts references
roggervalf f535033
test: fix repeat test case
roggervalf 0be0c0b
fix: flaky test
roggervalf bbf3afb
refactor: add moveJobFromPriorityToActive include
roggervalf a5de7da
test(priority): fix flaky test
roggervalf fab6b3a
chore: merge branch 'master' into better-priority
roggervalf 527eef5
refactor(priority): use priority counter key
roggervalf 9020413
chore: remove extra args
roggervalf acd0fef
refactor: update params
roggervalf 3d3a8e3
refactor(priority): use counter and priority as score
roggervalf ab92413
feat(queue): add removePriorityKey method
roggervalf 570ef51
refactor: reset priority counter when prioritized state is empty
roggervalf 6f20687
refactor(drained): consider prioritized length
roggervalf 8fc8fb7
refactor: change methods names updateData and removeDeprecatedPriorit…
roggervalf 4f069c6
chore: fix typo
roggervalf 070af22
refactor(push-back): re-add job at the head of same prioritized jobs
roggervalf 6d13180
chore: address comments
roggervalf 0a53a05
chore: merge branch 'master' into better-priority
roggervalf File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,6 +10,7 @@ | |
ARGV[2] job key | ||
ARGV[3] job id | ||
ARGV[4] lifo | ||
ARGV[5] timestamp | ||
|
||
Output: | ||
0 - OK | ||
|
@@ -25,20 +26,26 @@ local rcall = redis.call | |
--- @include "includes/getTargetQueueList" | ||
|
||
if rcall("EXISTS", jobKey) == 1 then | ||
local target = getTargetQueueList(KEYS[3], KEYS[1], KEYS[2]) | ||
|
||
local numRemovedElements = rcall("LREM", target, -1, jobId) | ||
if numRemovedElements > 0 then | ||
rcall("ZREM", KEYS[4], jobId) | ||
|
||
-- Standard or priority add | ||
if priority == 0 then | ||
-- LIFO or FIFO | ||
local pushCmd = ARGV[4] == '1' and 'RPUSH' or 'LPUSH'; | ||
rcall(pushCmd, target, jobId) | ||
else | ||
-- Priority add | ||
addJobWithPriority(KEYS[4], priority, target, jobId) | ||
local target, paused = getTargetQueueList(KEYS[3], KEYS[1], KEYS[2]) | ||
|
||
local pprefix = rcall("HGET", jobKey, "pprefix") | ||
|
||
local isPrioritized = pprefix and (rcall("ZREM", KEYS[4], pprefix .. ":" .. jobId) > 0) | ||
if isPrioritized then | ||
-- Priority add | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no need for the comment as the function is descriptive enough |
||
addJobWithPriority(KEYS[1], KEYS[4], jobKey, priority, paused, jobId, ARGV[5]) | ||
else | ||
local numRemovedElements = rcall("LREM", target, -1, jobId) | ||
if numRemovedElements > 0 then | ||
-- Standard or priority add | ||
if priority == 0 then | ||
-- LIFO or FIFO | ||
local pushCmd = ARGV[4] == '1' and 'RPUSH' or 'LPUSH'; | ||
rcall(pushCmd, target, jobId) | ||
else | ||
-- Priority add | ||
addJobWithPriority(KEYS[1], KEYS[4], jobKey, priority, paused, jobId, ARGV[5]) | ||
end | ||
end | ||
end | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
14 here will not always be correct. It would be better to use a regexp that catches the jobId: