Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deleting many items in the queue results in timeouts #16

Closed
benoist opened this issue Feb 18, 2021 · 1 comment · Fixed by #20
Closed

Deleting many items in the queue results in timeouts #16

benoist opened this issue Feb 18, 2021 · 1 comment · Fixed by #20
Assignees
@sharshenov
Milestone
0.2.0

Comments

@benoist
Copy link

benoist commented Feb 18, 2021

Hi,

Thanks for creating this awesome gem.

We've been using this for a while but when I upgraded to a recent version, I noticed that the deletion of jobs now also removes the lock. That's great, however when you delete a full queue of 1000+ items this causes timeouts on sidekiq admin and it actually doesn't seem to be clearing the queue.

My previous method was always to clear the queue and then unlock all items.
This method is much faster.
Perhaps a similar method can be used to unlock all items when you delete a full queue?
@sharshenov
Copy link
Member

Hey @benoist,
Sorry for the late response. I think there is no way to make the gem faster as queue name is not present in lock key and a queue may hold jobs of any class. Deleting a Sidekiq queue results "N+1"s of lock cleanup per job.
I'll try to make sidekiq patch to be optional, so in your case you could disable it completely and you could use your locks cleanup approach.

sharshenov added a commit that referenced this issue May 8, 2021
@sharshenov
Do not apply Sidekiq patch automatically
b8ab37d 
Deleting many items in the queue via Sidekiq Web UI results in timeouts because each job is being unlocked individually.
Since this change Sidekiq API patch should be applied explicitly in Gemfile:

```ruby
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
```

fixes #16
sharshenov added a commit that referenced this issue May 8, 2021
@sharshenov
Do not apply Sidekiq patch automatically
58787e7 
Deleting many items in the queue via Sidekiq Web UI results in timeouts because each job is being unlocked individually.
Since this change Sidekiq API patch should be applied explicitly in Gemfile:

```ruby
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
```

fixes #16
@sharshenov sharshenov mentioned this issue May 8, 2021
Merged
@sharshenov sharshenov self-assigned this May 8, 2021
sharshenov added a commit that referenced this issue May 9, 2021
@sharshenov
Do not apply Sidekiq patch automatically
748b51b 
Deleting many items in the queue via Sidekiq Web UI results in timeouts because each job is being unlocked individually.
Since this change Sidekiq API patch should be applied explicitly in Gemfile:

```ruby
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
```

fixes #16
@sharshenov sharshenov added this to the 0.2.0 milestone May 9, 2021
sharshenov added a commit that referenced this issue May 9, 2021
@sharshenov
Do not apply Sidekiq patch automatically
def4fb0 
Deleting many items in the queue via Sidekiq Web UI results in timeouts because each job is being unlocked individually.
Since this change Sidekiq API patch should be applied explicitly in Gemfile:

```ruby
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
```

fixes #16
sharshenov added a commit that referenced this issue May 9, 2021
@sharshenov
Do not apply Sidekiq patch automatically
408b691 
Deleting many items in the queue via Sidekiq Web UI results in timeouts because each job is being unlocked individually.
Since this change Sidekiq API patch should be applied explicitly in Gemfile:

```ruby
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
```

fixes #16
@hallelujah hallelujah mentioned this issue May 26, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sharshenov sharshenov

Labels
None yet
Projects
None yet
Milestone
0.2.0
Development

Successfully merging a pull request may close this issue.

Do not apply Sidekiq patch automatically veeqo/activejob-uniqueness
2 participants
@benoist @sharshenov