AWS Elasticache Redis Clusters OK?: Redis::CommandError (ERR SELECT is not allowed in cluster mode):

[lapygithub]

In development (MacOSX) the Rails/Sidekiq/Redis setup works correctly.

For QA/Stage/Prod, I've setup an AWS Elasticache Redis cluster (single master/single slave for automated failover).
In this environment, when calling the Sidekiq async worker, the following error is produced:

I, [2017-05-02T19:53:14.186415 #15767] INFO -- : Completed 500 Internal Server Error in 294ms
F, [2017-05-02T19:53:14.188203 #15767] FATAL -- :
Redis::CommandError (ERR SELECT is not allowed in cluster mode):
app/models/user.rb:1654:in `push_to_edm'

The stack trace just points to the .perform_async line listed below.

I feel like there is a Rails side configuration that I've missed and in searching found a few non-Heroku config blogs, but none describe anything but the standard Sidekiq Rails side code. See sidekiq.rb below. (BTW, the Redis db '/12' is confusing me too. ;o)

Does Sidekiq support Redis cluster or do you have to setup a non-clustered Redis?
And/Or: What do I need to correct?

Regards,
-MikeL

Calling code:

PushToEdmWorker.perform_async(self.identityid, edm_person_json, Rails.configuration.api_secret)

Ruby version: 2.1.4
Sidekiq / Pro / Enterprise version(s): Sidekiq

Please include your initializer and any error message with the full backtrace.

Sidekiq.remove_delay!

Sidekiq.configure_server do |config|
  config.redis = { url: "redis://#{REDIS_CONFIG['sidekiq_host']}:#{REDIS_CONFIG['sidekiq_port']}/12" }
end

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{REDIS_CONFIG['sidekiq_host']}:#{REDIS_CONFIG['sidekiq_port']}/12" }
end

Are you using an old version?
I believe we are using the latest 4.2.x release (since 5 has been released)

sidekiq (4.2.10)
  concurrent-ruby (~> 1.0)
  connection_pool (~> 2.2, >= 2.2.0)
  rack-protection (>= 1.5.0)
  redis (~> 3.2, >= 3.2.1)

Have you checked the changelog to see if your issue has been fixed in a later version?
I've searched using all the search terms I can think of...
https://github.com/mperham/sidekiq/blob/master/Changes.md

[mperham]

You need to change 12 to 0 in your initializer.

[lapygithub]

Thanks MikeP!
Made the change, but in the meantime configured a non-clustered Redis which all worked.

Simply changing my Redis environment variable to the clustered Redis now gives me this error at the same code location described above:

I, [2017-05-02T23:44:03.190516 #6350] INFO -- : Completed 500 Internal Server Error in 279ms
F, [2017-05-02T23:44:03.192181 #6350] FATAL -- :
Redis::CommandError (CROSSSLOT Keys in request don't hash to the same slot):
app/models/user.rb:1654:in `push_to_edm'

I started reading about Redis Hash Tags for clusters, but this fix my be in Sidekiq code...

Please let me know if you'd like me to submit a new issue.

Regards,
-MikeL

[mperham]

I don't know anything about the CROSSSLOT error. Maybe someone with Elasticache experience knows more.

[ryansch]

@lapygithub Are you using sidekiq-pro or sidekiq-ent? Some features are not compatible with Redis Cluster. Example: https://github.com/mperham/sidekiq/wiki/Reliability#scheduler

[lapygithub]

@ryansch : I haven't upgraded and also have not configured advanced features like scheduler. Configuration changes that I have made are to use a named queue (not default), to remove the delayed jobs conflict, port number 6381 and as simple of an async_perform that I could create to enqueue an http call out.

[l4u-co]

Just experimented the same problem ERROR: heartbeat: CROSSSLOT Keys in request don't hash to the same slot on a clustered AWS ElastiCache configuration.
No problem with standalone or replicated AWS ElastiCache configuration.

@lapygithub : did you solve or open a new issue on this point ?

[lapygithub]

Once I had it running non-clustered, I've just left it that way. If I get the opportunity to test a config change, I will, but that might be sometime in the future.

[trooster]

Same error here "heartbeat: CROSSSLOT Keys in request don't hash to the same slot".
@laurent-dazzl what was your solution? Or did you move away from a clustered AWS ElastiCache configuration as well?

[mperham]

Please see the Using Redis wiki page. I do not recommend using Cluster.

…

On Jul 14, 2018, at 05:27, Joris Trooster ***@***.***> wrote: Same error here "heartbeat: CROSSSLOT Keys in request don't hash to the same slot". @laurent-dazzl what was your solution? Or did you move away from a clustered AWS ElastiCache configuration as well? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[maniSHarma7575]

You need to change 12 to 0 in your initializer.

@mperham
Why 12 to 0?

Redis can support upto 0-15 databases