can it run on multiple servers?

[mgutz]

I ran the same hello example in two processes to test multi-server deployment

# terminal 
go run main.go -queues="hello" -concurrency=25
# another terminal
go run main.go -queues="hello" -concurrency=25
# queued up 100 messages
redis-cli -r 100 RPUSH resque:queue:hello '{"class":"MyClass","args":["hi","there"]}'

I expected the 100 messages to be processed equally between the two processes. The first process processsed all 100 messages. Does goworker support running on multiple servers?

[benmanns]

Yes! I think what you ran into is that by default there is a 5 second delay if goworker asks Redis for a job and there is none available. This is to prevent goworker from fetching from an empty queue 1000s of times per second and overloading your Redis server. Check out the -interval flag.

You can try your example again with a lower interval and a higher number of jobs:

# terminal 
go run main.go -queues="hello" -concurrency=25 -interval=0.1
# another terminal
go run main.go -queues="hello" -concurrency=25 -interval=0.1
# queued up 100 messages
redis-cli -r 10000 RPUSH resque:queue:hello '{"class":"MyClass","args":["hi","there"]}'