Sidekiq Enterprise Scheduled Jobs Not Showing up Consistently

[bobber205]

Ruby version: 2.3.1
Sidekiq / Pro / Enterprise version(s): Sidekiq 5.1.1 / Pro 4.0.1 / Ent 1.7.0

Please include your initializer and any error message with the full backtrace.

 Sidekiq.configure_server do |config|
    config.super_fetch!
    config.reliable_scheduler!
    puts "doing periodic"
    config.periodic do |manager|
      manager.register("*/1 * * * *", 'InstagramPollingManager', retry: 0, queue: 'default')
      manager.register("*/1 * * * *", "Chirpify::Workers::Utilities::FacebookStatusChecker", retry: 0, queue: 'default')

Are you using an old version? No
Have you checked the changelogs to see if your issue has been fixed in a later version? Yes

Not quite sure what is going on. The above initializer worked fine yesterday.
This morning I woke up to convert our much longer list of cron jobs that is in a different project

Now I have several more lines like this

      manager.register("*/5 * * * *", "Chirpify::Workers::FileSystemMetrics", retry: 0, queue: 'workers')
      manager.register("*/1 * * * *", "Chirpify::Workers::AWSHeartbeats", retry: 0, queue: 'workers')

And all I see is No periodic jobs found when going to /loops.
I was briefly running an empty config.periodic{} but I took that out and restarted all processed and it's still empty. Where can I look to debug this?

[mperham]

It sounds like you are running several different apps using the same Redis. You can't do this; each Redis keyspace belongs to one app, and one Sidekiq process will be elected leader to manage the periodic jobs for that app. If you have multiple apps, that's multiple leaders and they will stomp on each other's cron jobs. Easiest thing to do is to use a different Redis DB for each app, where DB is a number between 0-15:

config.redis = { db: 1 }

[bobber205]

I can easily do that and thank you for the fast response!

[bobber205]

That fixed my issue!
I do have one small suggestion: The other middleware we were using allowed us to run a job on demand if need be. It also allowed us to pause/enable cron jobs. Is it possible that the feature could make it's way into the official Web UI?
I know most of that is technically doable from a command line script or w/e but that doesn't help when you're on the road and I need to run a job from my phone. :)

[bobber205]

We often queue up jobs (due to not having namespace access in lots of various projects) like this

Sidekiq::Client.new.push(
           'queue' => 'workers',
           'class' => 'Chirpify::Workers::GenericLoggerWorker',
           'args' => ["debug_log", payload.to_json])

I noticed once I moved app 2 to {db: 1} like you suggested the apps just sat in db 0 's queue and were never processed. Is there a way an easy queue up jobs like this to the right redis keyspace?

[mperham]

That's sharding. App A will have a pool of connections to Redis B and tell Sidekiq::Client to push using that pool. https://github.com/mperham/sidekiq/wiki/Sharding

[bobber205]

This is quite a change I am not prepared to do right now -- is there anyway for the time being to do just tell Sidekiq that App A should always be the leader?

[mperham]

You can restructure all your periodic jobs to be "owned" by App A.

[bobber205]

Ah I can do that. As long as only one app is in charge of scheduling and setting up the cron jobs it should be fine? It's only an issue when there is multiple sets?

[mperham]

Actually I'm not sure that will work.

[mperham]

The issue is the leadership; App B will still try to become leader. There's no way to force leadership onto a particular process because that's a SPOF. I don't think having multiple apps cohabitate a single Redis is possible currently.

[bobber205]

We've had multiple apps in a single redis db for years now and it's worked ok. It's only been an issue with this cron feature (so far)

[mperham]

Yep, periodic jobs is the main feature that uses leadership. There's no problem using your current cron plugin in the short term; you can migrate to Enterprise's periodic feature over time or chose to stay with the plugin.

[bobber205]

The current cron middleware doesn't seem to be compatible with this version of sidekiq.

[mperham]

Then the next approach would be to have every periodic job registered into every app so no matter which app becomes leader, it will create the jobs for all apps.

[bobber205]

Oh I can easily do that! thanks!