How to test that sidekiq.yml is *correct*?

[hakunin]

Hello, I had a "typo" in my sidekiq.yml recently which caused the rest of my jobs not being scheduled and had to redo all the oauth connections for the users.

:scheduler:
  :schedule:
    gather_standups:
      every: '0 0 * * * *' # the typo
      #cron: '0 0 * * * *' # the correct value
      class: RunStandups
    heartbeat:
      every: '1 minute'
      class: HeartbeatWorker
    refresh_tokens:
      # the benefit here is spacing out the refreshes
      every: '15 minutes'
      class: RefreshTokenJob

I am trying to make sure this doesn't happen again and so I wanted to test that the config does not result in any jobs skipped (missing at/every/cron..), or outright failures.

The error in my typo case was:

ArgumentError
not a duration "0 0 * * * *"

fugit (1.8.1) lib/fugit/duration.rb in do_parse at line 44
rufus-scheduler (3.9.1) lib/rufus/scheduler/util.rb in parse_duration at line 86
rufus-scheduler (3.9.1) lib/rufus/scheduler/util.rb in parse_in at line 31
rufus-scheduler (3.9.1) lib/rufus/scheduler/jobs_repeat.rb in initialize at line 173
rufus-scheduler (3.9.1) lib/rufus/scheduler.rb in new at line 724
rufus-scheduler (3.9.1) lib/rufus/scheduler.rb in do_schedule at line 724
rufus-scheduler (3.9.1) lib/rufus/scheduler.rb in every at line 206
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in new_job at line 258
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in block in load_schedule_job at line 130
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in each at line 123
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in load_schedule_job at line 123
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in block in load_schedule! at line 101
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in each at line 99
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/scheduler.rb in load_schedule! at line 99
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/manager.rb in start at line 24
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler/sidekiq_adapter.rb in start_schedule_manager at line 41
sidekiq-scheduler (5.0.3) lib/sidekiq-scheduler.rb in block (2 levels) in <main> at line 21
sidekiq (7.0.2) lib/sidekiq/component.rb in block in fire_event at line 58
sidekiq (7.0.2) lib/sidekiq/component.rb in each at line 57
sidekiq (7.0.2) lib/sidekiq/component.rb in fire_event at line 57
sidekiq (7.0.2) lib/sidekiq/cli.rb in run at line 105
sidekiq (7.0.2) bin/sidekiq in <top (required)> at line 31

So I tried writing a test in all different ways but none of them produced the error above.

For example this one fails with:

       expected no Exception, got #<NoMethodError: undefined method `[]' for nil:NilClass
           @frequency = Rufus::Scheduler.parse(opts[:frequency] || 0.300)

But it fails regardless of that "typo"..

RSpec.describe 'Sidekiq Scheduler config' do
  let(:config_path) { Rails.root.join('config', 'sidekiq.yml') }
  let(:config) { YAML.load_file(config_path) }

  it 'has valid cron syntax for scheduled jobs' do
    scheduled_jobs = config.dig(:scheduler, :schedule)

    scheduled_jobs.each do |job_name, job_config|
      expect { 
        ap({ job_name => job_config })

        SidekiqScheduler::Scheduler.instance.load_schedule_job(
          job_name,
          job_config
        )
      }.not_to raise_error
    end

  end
end

Ultimately I would like to validate the format is correct without having to actually create those jobs in the redis.

[hakunin]

Another stab at this, this finally fails with the same error.
A bit dirty (knows too much) but good start for now.

RSpec.describe 'Sidekiq Scheduler config' do
  let(:config_path) { Rails.root.join('config', 'sidekiq.yml') }
  let(:config) { YAML.load_file(config_path) }
  let(:jobs) { config.dig(:scheduler, :schedule) }

  let(:scheduler_config) { SidekiqScheduler::Config.new(sidekiq_config: {}) }
  let(:scheduler) { SidekiqScheduler::Scheduler.new(scheduler_config) }

  # to silence the logger
  let(:sidekiq_config) { Sidekiq.instance_variable_get('@config') }

  before do
    # silence the logging
    @original_logger = sidekiq_config.logger
    sidekiq_config.logger = ::Logger.new("/dev/null")

    # some setup was skipped and this was nil
    scheduler.instance_variable_set('@scheduled_jobs', {})

    # prevent jobs from being actually enqueued
    allow(scheduler).to receive(:idempotent_job_enqueue)
  end

  after do
    sidekiq_config.logger = @original_logger
  end

  it 'loads the sidekiq configuration for each job without errors' do
    jobs.each do |job_name, job_config|
      expect {
        scheduler.load_schedule_job(job_name, job_config)
      }.not_to raise_error
    end
  end

  it 'test that all classes exist' do
    jobs.each do |job_name, job_config|
      expect { job_config["class"].constantize }.not_to raise_error
    end
  end
end

This also checks that the classes are "loadable" since Sidekiq Scheduler doesn't fail on that when I stubbed the enqueue method.

[marcelolx]

@hakunin I haven't worked with rspec and sidekiq-scheduler for some time now, so I don't think I am the best person to suggest how to test a sidekiq.yml — what I can say is that in the previous places we used it, we didn't test it at all, because we usually had some monitoring on the scheduled jobs to make sure they are running when we expected them to run (Sentry Crons is an example)

All that said, I set up discussions on this repo, so things like this can be discussed by the community there https://github.com/sidekiq-scheduler/sidekiq-scheduler/discussions

[marcelolx]

Feel free to open a discussion there with all this context @hakunin!