Use adapters for the link and page queues #37

Closed
wants to merge 12 commits into
from

Conversation

Projects
None yet
4 participants

For now just the default (Ruby Queue class) and Redis. All tests pass.

lib/anemone/queue/redis.rb
+ end
+
+ def <<(job)
+ id = @redis.incr("#{@prefix}:counter")
@EvilScott

EvilScott Oct 24, 2011

Use Redis List instead of multiple keys http://redis.io/commands#list

lib/anemone/queue/redis.rb
+ module Queue
+ class Redis
+
+ def initialize(queue_type, opts = {})
@EvilScott

EvilScott Oct 24, 2011

Use self.hash (get rid of negatives?) instead of passing in queue_type for uniq key

lib/anemone/queue/redis.rb
+ job.each { |k,v| @redis.hset("#{@prefix}:#{id}", k, v) }
+ end
+
+ def deq
@EvilScott

EvilScott Oct 24, 2011

Keep a key of number of waiting threads. incr at the beginning of this function, decr at the end.

lib/anemone/queue/redis.rb
+ end
+
+ def size
+ keys.count
lib/anemone/queue/redis.rb
+ end
+
+ def empty?
+ keys.count == 0
lib/anemone/queue/redis.rb
+ end
+
+ def num_waiting
+ keys.count
@EvilScott

EvilScott Oct 24, 2011

Return key of num waiting threads to_i

lib/anemone/queue.rb
+module Anemone
+ module Queue
+
+ def self.Default(*args)
@EvilScott

EvilScott Oct 24, 2011

Rename to Basic

lib/anemone/core.rb
+ if Queue.methods.include? !@opts[:link_queue_adapter]
+ raise "Unknown link queue adapter #{@opts[:link_queue_adapter]}"
+ end
+ link_queue = Queue.send(@opts[:link_queue_adapter], :link, @opts[:link_queue_options])
@EvilScott

EvilScott Oct 24, 2011

link_queue = @opts[:link_queue] || Anemone::Queue.Default

lib/anemone/core.rb
+ # use default queue adapter for link queue
+ :link_queue_adapter => :Default,
+ # link queue options sub-hash
+ :link_queue_options => {},
@EvilScott

EvilScott Oct 24, 2011

No need for options here, as they are passed into Queue.Redis

Scott Reis added some commits Oct 25, 2011

use redis list instead of multiple keys for queue, rename Default to …
…Basic, update tests, and remove default adapter options (since they will be passed into the adapters explicitly
lib/anemone/queue/redis.rb
+ end
+
+ def <<(job)
+ @redis.lpush(@list,job.to_json)
@EvilScott

EvilScott Oct 28, 2011

Lazy load Thread local redis connection

lib/anemone/queue/redis.rb
+ end
+
+ def deq
+ json = @redis.rpop(@list)
@EvilScott

EvilScott Oct 28, 2011

Lazy load redis here too

lib/anemone/queue/redis.rb
+ def clear
+ @redis.del(@list, @waiting)
+ end
+
@EvilScott

EvilScott Oct 28, 2011

def redis
Thread[:redis] ||= ::Redis.new(@opts)
end

lib/anemone/queue/redis.rb
+ class Redis
+
+ def initialize(opts = {})
+ @redis = ::Redis.new(opts)
@EvilScott

EvilScott Oct 28, 2011

Store opts in instance var

Contributor

bernd commented Jan 20, 2012

I've been using this since November. Works fine for me.

When I understand this correctly, this is to switch the in-memory queue to Redis and other memory control patches?

After 4 years, it may be time to let this die 💀

@EvilScott EvilScott closed this Aug 28, 2015

Contributor

brutuscat commented Aug 28, 2015

@EvilScott maybe you could take a look at my fork and submit there instead? https://github.com/brutuscat/medusa

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment