Remember dynamic schedule on deploys

[sineed]

Hi! I just started using your gem. I found it very useful because it allows to make dynamic schedules.

I also found that it does not remember the last schedule if I restart my Sidekiq process. It usually happens on deployment.

I thought about hooking into deployment process to create a file and write to it the latest Sidekiq.get_schedule value. After successful start of Sidekiq I need to do the same thing as described here.

So in the end I will have the following configuration for Sidekiq:

require "sidekiq"
require "sidekiq/scheduler"

RELATIVE_PATH = "path_to_schedule_file.yml"

def get_schedule
  schedule   = Sidekiq.get_schedule
  next_times = Redis.current.hgetall(Sidekiq::Scheduler.next_times_key)

  next_times.each do |job_key, first_at|
    next unless schedule[job_key]
    schedule[job_key]["every"] << { "first_at" => first_at }
  end

  schedule
end

Sidekiq.configure_server do |config|
  config.on(:startup) do
    file_path = File.expand_path(RELATIVE_PATH, __FILE__)
    if File.exist?(file_path)
      File.open(file_path) do |f|
        Sidekiq.schedule = YAML.load_file(f)
        Sidekiq::Scheduler.reload_schedule!
      end
    end
  end

  config.on(:quiet) do
    file_path = File.expand_path(RELATIVE_PATH, __FILE__)
    file = File.new(file_path, "w+")
    file << get_schedule.to_yaml
    file.close
  end

  config.on(:shutdown) do
    file_path = File.expand_path(RELATIVE_PATH, __FILE__)
    file = File.new(file_path, "w+")
    file << get_schedule.to_yaml
    file.close
  end
end

This code is wrong, it just shows the basic idea.

Maybe my findings can be useful for someone else.

--
UPD. I've updated configuration example, now it is correct and is used in my project. I want to point that for me it is also important to remember the next time a job will be enqueued

[ladiadeniran]

Nice one keeping this out here. Our workaround involved writing a job that runs on starting sidekiq.

[rrrcompagnoni]

I think you can set Sidekiq.schedule = Sidekiq.get_all_schedules in your config/initializers/sidekiq.rb but I'm not sure about the effects of that.

[ladiadeniran]

Alright thanks @rrrcompagnoni I will try that out too.

[glebmikulko]

@snmgian any news about it?

[bpo]

I think this comment handles this use case (i.e. you can just run Sidekiq.reload_schedule! rather than serializing back and forth to YAML).

[Floppy]

It ceratinly seems like the scheduler should load from Redis on boot if dynamic, the line at

sidekiq-scheduler/lib/sidekiq-scheduler/scheduler.rb

Line 85 in 9da5f13

# Load schedule from redis for the first time if dynamic

suggests as much. But it doesn't seem to be happening.

The code there already runs Sidekiq.reload_schedule! for dynamic schedules, but somewhere lower in the stack it's not having the desired effect. Definitely feels like there's a bug in there somewhere...

[Floppy]

Found it. This line is clearing the "schedules" key out of redis on startup:

sidekiq-scheduler/lib/sidekiq-scheduler/manager.rb

Line 16 in 9da5f13

Sidekiq.schedule = config.schedule if @scheduler_instance.enabled

I assume that shouldn't happen if the schedule is set to dynamic; I'll open a PR.