Skip to content

How to run separate workers per queue on Heroku with SolidQueue? #575

@drale2k

Description

@drale2k

I'm migrating from GoodJob to SolidQueue and trying to replicate a setup where I ran one dyno per queue via a Heroku Procfile. Here's what I had before:

good_job_fast_queue: bundle exec good_job --queues="fast:3"
good_job_queues: bundle exec good_job --queues="default,fast:5;*:5"
good_job_whenever: bundle exec good_job --queues="whenever:10"
good_job_meilisearch_queue: bundle exec good_job --queues="meilisearch:10"
good_job_meilisearch_second_queue: bundle exec good_job --queues="meilisearch:10"
good_job_meilisearch_index_queue: bundle exec good_job --queues="meilisearch_index:15"

How can I do the equivalent with SolidQueue on Heroku?

  • queue.yml defines multiple workers, but Heroku only starts dynos defined in the Procfile.
  • Is there a way to start a SolidQueue worker for a specific queue (e.g. just meilisearch) so I can control each dyno independently?

I have my queue.yml setup properly but on heroku this does not spawn a separate process (dyno):

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 100
  workers:
    - queues: 'fast'
      threads: 3
      polling_interval: 0.1
      processes: 1
    - queues: 'whenever'
      threads: 10
      polling_interval: 1
      processes: 1
    - queues: 'meilisearch'
      threads: 10
      polling_interval: 1
      processes: 2
    - queues: 'meilisearch_index'
      threads: 10
      polling_interval: 1
      processes: 1
    - queues: 'default'
      threads: 5
      polling_interval: 1
      processes: 1
    - queues: '*'
      threads: 5
      polling_interval: 1
      processes: 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions