HykuAddons: Background Job Processing with Sidekiq

Hyku Addons uses Sidekiq to process a variety of background jobs. When new customers are first bulk importing their works it is usual practice to switch on the Import Mode feature. This uses two different queues to run imports and the jobs they spawn, viz <<tenant_name>>_import_<<queue_name>>. For example, when Import Mode is turned on and an import is run the jobs which create collections and save works will be run on <<tenant_name>>_import_import. If the CSV file includes the doi_status_when_public column, when the works are saved a job will be enqueued on <<tenant_name>>_import_default to mint DOIs.

Local Development

The easiest way to ensure sidekiq is working for your local tenant is to add queues to the sidekiq.yml in the internal_test_hyku. For a tenant called demo add the following lines to the yaml:

:queues:
  - default
  - import
  - export
  - demo_import_import
  - demo_import_default

Web Interface

You can monitor workers and jobs from the Sidekiq Web Interface. Click here to access this on AH: https://dashboard.demo.ubiquityrepo-ah.website/sidekiq

Note that this is only available to logged in admin users. This is configured in the routes file:

  authenticate :user, lambda {|u| u.roles_name.include? 'admin' } do
    mount Sidekiq::Web => '/sidekiq'
  end

Deployment

Workers are not updated and restarted as part of the deployment process. This will need to be done manually. Please see this part of the wiki for details: Tenant Import Works