Lost keys when using job.progress(...)

[carlos870]

Description

The job.progress method is not checking if the job key actually exists before updating the job's progress, which means we could end up with with an empty key left in Redis, only with the updated progress.
If the job config uses a custom jobId (based on business logic - userId), this could become problematic, since no jobs for that user would be added anymore.

A simple solution that I tried and it seems to fix the issue, was to wrap the LUA script updateProgress (HSET/PUBLISH) inside a Redis EXISTS command.

This occurred in a scenario where we get the job (getJob()), and then try to update it's progress, and sometimes the job was already removed. We are using setting removeOnComplete.

Bull version

4.4.0

[manast]

Thanks for the report. Yes, this sounds like an oversight from my part, a job progress should only be possible to perform while a job is in active state and if the caller owns the lock.

[carlos870]

Thanks for looking into it!

Meanwhile I also noticed the same behaviour for the job.update action.

[manast]

For job.update I do not think it is necessary to check for a lock, since usually this method is used outside of the processor, on the other hand it could require the job to exist in the first place.

[roggervalf]

It's fixes in this pr #2730