Skip to content

Commit

Permalink
Improves stream trimming performance by approximating
Browse files Browse the repository at this point in the history
Allows the stream to grow a small amount beyond the maximum backlog length until a full macro node can be removed from the radix tree, which is significantly more efficient than being precise and editing nodes. This means we will use an amount of storage equivalent to a few tens of messages more than the maximum backlog length we specify, but throughput will be higher.
  • Loading branch information
benlangfeld committed Nov 6, 2018
1 parent 0c7fea1 commit 7aa0c16
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions lib/message_bus/backends/redis_streams.rb
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,10 @@ def expire_all_backlogs!
local backlog_id = redis.call("INCR", backlog_id_key)
local payload = string.format("%i|%i|%s", global_id, backlog_id, start_payload)
redis.call("XADD", backlog_key, "MAXLEN", max_backlog_size, string.format("0-%i", backlog_id), "payload", payload)
redis.call("XADD", backlog_key, "MAXLEN", "~", max_backlog_size, string.format("0-%i", backlog_id), "payload", payload)
redis.call("EXPIRE", backlog_key, max_backlog_age)
redis.call("XADD", global_backlog_key, "MAXLEN", max_global_backlog_size, string.format("0-%i", global_id), "payload", payload)
redis.call("XADD", global_backlog_key, "MAXLEN", "~", max_global_backlog_size, string.format("0-%i", global_id), "payload", payload)
redis.call("EXPIRE", global_backlog_key, max_backlog_age)
return backlog_id
Expand Down

0 comments on commit 7aa0c16

Please sign in to comment.