Running Light Service organizers and actions in jobs

Sometimes you need to run your organizers inside of an async job system like Sidekiq. It can be a little confusing to figure out exactly how they could work together.

One solution would be to make extend your Organizer to be capable of running as an asynchronous job. Doing this also removes the need to have a corresponding _job.rb file (unless you still want to have one lying around).

In a nutshell, you need to handle the following:

• Making sure your job can take primitive values, and turn them into rich objects that your organizer relies upon.
• Making sure that should the organizer fail, the job system has the opportunity to retry the job if you so desire.
• Make sure the organizer works as normal when called synchronously

class MyReportOrganizer < ApplicationJob # NOTE: Inherits from ApplicationJob
  extend LightService::Organizer         # as well as extending ls-org

  queue_as :default                      # Specify the queue

  def self.call(payload)
    with(date_range: payload[:date_range], email: payload[:email])
      .reduce(actions)
  end

  def self.actions
    [
      Reports::CollectRawData,
      Reports::WriteSpreadsheet,
      Reports::EmailReport,
    ]
  end

  # Implement a perform action that takes primitive variables which are typically safer
  # to pass around when using systems like Sidekiq
  def perform(start_date_string, end_date_string, email)
    date_range, email = hydrate(start_date_string, end_date_string, email)

    outcome = MyReportOrganizer.call(date_range: date_range, email: email) # Call the organizer 

    raise RuntimeError("Unable to process report.") if outcome.failure?    # Raise an exception to force a retry if .failure?

    true
  end

  private
  
  # Implement a hydrate method that can take all the primitive arguments the job receives
  # via perform, and turn them into rich objects the organizer uses.
  def hydrate(start_date_string, end_date_string, email)
    start_date = Date.parse(start_date_string)
    end_date   = Date.parse(end_date_string)
    date_range = (start_date..end_date)

    [date_range, email]
  end
end

So, what we've got here is a hybrid organizer/job that allows us to call it in the traditional manner from a controller, rake task, etc with rich objects and have it run synchronously, or call it like so:

MyReportOrganizer.perform_later(a_bunch, of_primitive, variables_here)

and it will take those primitives (which some job systems require), hydrate them into rich objects that are required for synchronous calling, and then run the organizer, failing it with an exception (to ensure a retry) if the job is a failure.