Skip to content

Locking no longer functions correctly #40

Closed
KieranP opened this Issue Mar 4, 2013 · 32 comments
@KieranP
KieranP commented Mar 4, 2013

I upgraded from v0.4.1 to v0.4.3 and found that the job locking no longer functions correctly (using Rails 3.2.12 and Postgresql 9.1).

I have certain jobs setup to requeue themselves upon completion. Multiple workers pick up the same job, run it, and queue the same job multiple times, resulting in one job becoming 2-3 jobs. They in turn get picked up and run and create more.

As it stand, I have about 7 jobs queued when only 1 should be there. It worked great with 0.4.1 and the issue seems to have been introduced in these commits:

b015618
0cb4496

For now, I have rolled back to v0.4.1 and the bug has gone away.

@jeyb
jeyb commented Mar 4, 2013

Running into the same problem with MySQL.

In addition, we are seeing Delayed::Job record not found exceptions as one job finished processing the job while the other is attempting to lock. This literally crashes the worker process.

@pa657
pa657 commented Mar 6, 2013

Same issue.

@jensblond

+1

@schmierkov

+1

@n8gard
n8gard commented Mar 7, 2013
@martoche

+1

@n8gard
@maletor
@KieranP
KieranP commented Mar 19, 2013

Bump. This is a fairly important issue that needs to be fixed ASAP because it can lead to all sorts of havoc, such as email users multiple times for the same thing (think credit card payment notifications being sent out 3-5 times!).

@fionaom
fionaom commented Mar 25, 2013

+1

@kwent
kwent commented Mar 26, 2013

+1 !

@snusmu
snusmu commented Mar 26, 2013

+1474

thats the amount of dj's created overnight after we updated dj_ar to 0.4.2

@testingbot

+1

So the fix is to lock the version to 0.4.1 in the Gemfile?
If I do "gem list" I don't even see delayed_job_active_record, so not sure how I can uninstall the 0.4.3 version.

@brupm
brupm commented Mar 26, 2013

+1

@vedanova

+1

@Cr1stal
Cr1stal commented Mar 28, 2013

+1

@donbobka

+1

@pashpashkin

+1

@mmoldavan

+1

@gaffneyc
Collective Idea member

Would someone be willing to put together a pull request to fix this?

@gaffneyc
Collective Idea member

@martoche Understandable that you don't feel comfortable enough with the codebase to make a pull request. And it really sucks that this has been outstanding as long as it has been.

That said... if anyone following this ticket is willing to do the work to revert those commits and make a pull request we can likely get a new gem cut today and the old one yanked. At the very least we'll get the bad gem yanked today.

@gaffneyc
Collective Idea member

And can anyone confirm that this was introduced in 0.4.2?

@albus522 albus522 referenced this issue Mar 28, 2013
Merged

Fix job locking #46

@sferik
sferik commented Apr 1, 2013

I'm sorry it has taken so long for me to get to this issue. I’ve merged @albus522’s patch into master and cherry-picked the relevant changes into the 0-4-stable branch.

Can someone please confirm the fix by replacing the delayed_job_active_record line in their Gemfile with:

gem 'delayed_job_active_record', :git => 'git://github.com/collectiveidea/delayed_job_active_record.git', :branch => '0-4-stable' # or 'master' for Rails 4

After this fix is confirmed, I will release gem versions 0.4.4 and 4.0.0.beta2. Unless the patch introduces new issues, these gems should be released later today.

@martoche
martoche commented Apr 2, 2013

There's no way to test this? I don't feel confortable with pushing this in production to test it.

@sferik
sferik commented Apr 2, 2013

I've released gem versions 0.4.4 and 4.0.0.beta2.

@sferik sferik closed this Apr 2, 2013
@justincampbell

This is fixed for me in 4.0.0.beta2

Thanks!

@jamsi
jamsi commented Apr 3, 2013

Adding this to my Gemfile + bundle install fixed the issue for me.

gem 'delayed_job_active_record', '0.4.1'
@sferik
sferik commented Apr 3, 2013

@jamsi Can you please try version 0.4.4 (released this morning) and let me know if that works for you?

@kwent
kwent commented Apr 3, 2013

Seems OK with 0.4.4 for me. Thanks !

@jamsi
jamsi commented Apr 3, 2013

@sferik 0.4.4 seems to be ok too. I'm just glad I caught this before I pushed to production. It's a hard thing for rspec to pickup =\

Many thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.