Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggested configuration changes #328

Open
mhenrixon opened this issue Aug 21, 2018 · 3 comments
Open

Suggested configuration changes #328

mhenrixon opened this issue Aug 21, 2018 · 3 comments

Comments

@mhenrixon
Copy link
Owner

mhenrixon commented Aug 21, 2018

Been thinking a little on the way the worker is configured. There was a suggestion on how to rather than use the UntilAndWhileExecuting lock that two separately configured workers with different conflict strategies would be appreciated. The only two ways I can think of doing this would be something like below:

### SUGGESTION 1
class ComposedWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :composed, retry: 0, backtrace: true
  lock until_executing: { on_conflict: :reject, expiration: 5.seconds },
       while_executing: { on_conflict: :update, expiration: 10.seconds }
end

### OR 
class ComposedWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :composed, retry: 0, backtrace: true
  lock until_executing: { on_conflict: :reject, expiration: 5.seconds }
  lock while_executing: { on_conflict: :update, expiration: 10.seconds }
end

class UntilExecutingWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :critical, retry: 0, backtrace: true
  lock until_executed: { on_conflict: :log, timeout: 7 }
end

class WhileExecutingWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :default, retry: 0, backtrace: true
  lock while_executing: { on_conflict: :update, expiration: 6.seconds }
end

### SUGGESTION 2

class ComposedWorker
  include Sidekiq::Worker
  sidekiq_options lock: [
    { type: :until_executing, on_conflict: :reject, expiration: 5.seconds },
    { type: :while_executing, on_conflict: :update, expiration: 10.seconds }
  ]
end

class UntilExecutingWorker
  include Sidekiq::Worker
  sidekiq_options lock: { type: :until_executed, on_conflict: :log, timeout: 7 }
end

class WhileExecutingWorker
  include Sidekiq::Worker
  sidekiq_options lock: { type: :while_executing, on_conflict: :update, expiration: 6.seconds }
end

They both have their advantages and drawbacks. Personally I lean towards the first suggestion which I feel provide the most intuitive way of configuring the worker. The class level lock method would basically just add whatever I want to the sidekiq_options hash but I like taking the gem in that direction since someone just discovered that first of all, no conflict strategy is used for the until_executing part of until_and_while_executing and secondly they ran into some problems with multiple jobs being pushed.

Anyone have any feedback on this? Maybe change it for v7.

@tahanson
Copy link

My vote would be something like your first suggestion. Maybe something like this:

class ComposedWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :composed, retry: 0, backtrace: true
  sidekiq_lock :until_executing, on_conflict: :reject, expiration: 5.seconds
  sidekiq_lock :while_executing, on_conflict: :update, expiration: 10.seconds
end

I assume this change would mean only 3 types of locks are needed?
(until_executing, while_executing and until_executed)

What about changing the lock names?

  • while_queued (formerly until_executing)
  • while_queued_and_executing (formerly until_executed)
  • while_executing (same as before)

@mhenrixon
Copy link
Owner Author

Great suggestions @tahanson ! I'm so glad I've awaited some feedback on this

@sdhull
Copy link

sdhull commented Mar 12, 2024

Love this idea! And I like @tahanson's suggestions for making the lock names more consistent / easier to understand

I'd love the class method name to be more-obviously related to this gem, eg

class ComposedWorker
  include SidekiqUniqueJobs::UniqueWorker
  sidekiq_options queue: :composed, retry: 0, backtrace: true
  sidekiq_unique_jobs lock: :until_executing, on_conflict: :reject, expiration: 5.seconds
  sidekiq_unique_jobs lock: :while_executing, on_conflict: :update, expiration: 10.seconds
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants