Skip to content

Loading…

Allow to use Socket instead of TCP #228

Closed
fred opened this Issue · 8 comments

5 participants

@fred

Is it possible to allow Sidekiq to use Redis with Socket?
Redis-rb allows to use socket like this:

redis = Redis.new(:path => "/tmp/redis.sock")

The benchmarks at redis site shows that using sockets yields up to 50% performance improvements.

Extracted from http://redis.io/topics/benchmarks

"When the server and client benchmark programs run on the same box, both the TCP/IP loopback and unix domain sockets can be used. It depends on the platform, but unix domain sockets can achieve around 50% more throughput than the TCP/IP loopback (on Linux for instance). The default behavior of redis-benchmark is to use the TCP/IP loopback."

Another reason is that using TCP you will see a lot of connections in wait/close state, such as:

# netstat -na | grep 6379

...
tcp 0 0 127.0.0.1:9105 127.0.0.1:6379 ESTABLISHED
tcp 1 0 127.0.0.1:9681 127.0.0.1:6379 CLOSE_WAIT
....

(edited: not twice as fast, but 50% faster)

@jc00ke
Collaborator

Currently it is not possible because Sidekiq::RedisConnection.build_client explicitly builds with a :url. I'm not sure if we'd accept a patch for sockets because they are only accessible locally. @mperham would have to give his blessing.

@fred

right, but I think both build_client and create methods can be easily modified to accept either a URL or PATH on the options.

I will do some experiments on my fork.

@mperham
Owner

Real world systems would need to scale Sidekiq to lots of machines before Redis networking overhead becomes a problem. Since Unix sockets only work on localhost, this is not an worthwhile feature.

@mperham mperham closed this
@tmak

I want to use sockets because it's just simpler in some cases. You don't have to worry about used ports and it's easier to setup a local redis installation which is just installed in the Rails application folder for simplicity. Currently i have the problem that i want to run two instances of the same Rails application on the same server, the production environment and a staging environment. With sockets it would be possible to use one redis configuration file and just use a relative path for the redis server socket. Right now i have to use two redundant redis configuration files to prevent conflicts with the redis server port. What do you think?

@mperham
Owner

Sidekiq defers all connection management to the Redis driver now that we just use the :url parameter. I think you can use Unix sockets with it, just figure out the correct URL, maybe this?

config.redis = { :url => 'unix:/tmp/redis.sock' }
@tmak

You are right, that works. Sorry, I should have thought of that.

For me that's fine, but with the :url parameter you can only set the path. That way it's not possible to set for example the database id for a unix socket connection.
See https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L322

Why don't you just pass the options hash for the Sidekiq Redis connection to the real Redis connection?

@betelgeuse
Collaborator

When I started with sidekiq I also expected there to be a way to pass the options down to the redis driver. Adding a support for that gets my vote but let's see what @mperham says. @tmak regardless of where sidekiq goes it could be a worthwhile addition to redis-rb to support urls like unix:///foo/bar?db=1 for unix sockets.

@fred

I had to fix the permissions on the socket file
I added access for everyone to read and write:

chmod o+rw /var/run/redis/redis.sock

Assuming the server is secured and isolated.

@benubois benubois referenced this issue in feedbin/feedbin
Closed

Redis error #120

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.