AbortException leaves an orphaned job file in the queue

[0xDEC0DE]

Spinach tasks with retries set do not clean themselves up completely if the job throws an AbortException. This is easiest to demonstrate with the RedisBroker:

Steps to reproduce

• Run the following test case:

from spinach import AbortException, Engine, RedisBroker
from redis.client import Redis

redis = Redis(host=YOUR_REDIS_HOST)
spin = Engine(RedisBroker(redis))

@spin.task(name='fail', max_retries=5)
def fail():
    raise AbortException

spin.schedule(fail)
spin.start_workers(stop_when_queue_empty=True)

• On the Redis, run KEYS spinach/_running*

Expected result

(empty array)

Actual behavior

The job file remains in the queue, e.g.:

1) "spinach/_running-jobs-on-broker-bcf5eb0c-f3ed-4270-afb8-d44e02b8bbad"

Errata

Tasks that do not set max_retries appear to clean themselves up properly.

[NicolasLM]

I confirm that this is a bug.

[bigjools]

I had an initial look to see if I can figure out the problem and didn't get anywhere. We could try setting job.retries = job.max_retries to force the cancellation instead of job.max_retries=0 and see if that helps? Something must be causing
self._broker.remove_job_from_running(job) from getting called in the _result_notifier_func.

[bigjools]

Anyway it would be nice to get this fixed and then we can do a release with the other change I just added.