Batch job: fail silently for X number of times

[helderasmoreira]

👋

I'd like to have Rails' ActiveJob#retry_on behaviour in a job that I use as part of a batch.

For reference, what this method does is:

Catch the exception and reschedule job for re-execution after so many seconds, for a specific number of attempts. If the exception keeps getting raised beyond the specified number of attempts, the exception is allowed to bubble up to the underlying queuing system, which may have its own retry mechanism or place it in a holding queue for inspection.

This seems particularly useful when jobs might fail for transient exceptions connecting to 3rd party APIs for instance.
I'd like them to retry a few times without notifying me and be notified so that I can look into it if they fail to succeeded within the X number of attempts.

I'm avoiding the retry_on directly because in the documentation it is stated explicitly that Batches don't work well with Active Job. I suspect that this exact "retry" behaviour might be part of the problem: ActiveJob makes it so that jobs don't really fail (raise an exception) but rather get re-scheduled, which might throw off batch "tracking".

So with this in mind, two questions:

• is there a way with Sidekiq native jobs to achieve this behaviour in a batch job, without breaking the tracking?
• would this be possible, again using Sidekiq native jobs, if the job is not part of any batch logic?

[mperham]

Most error services have some callback or option for this logic. If you don’t want your error service to contact you, configure it.

https://docs.honeybadger.io/lib/ruby/getting-started/ignoring-errors/#ignore-programmatically

[helderasmoreira]

Thank you for the quick reply 🙏

I understand that ignoring the exception "client" side could be possible (depending on your error provider) but achieving the desired behaviour — ignore specific exception from specific job, only until specific number of retries on a specific job — would make it a bit more complex I feel.

Which is why I'd like to see if support for this natively was present.

[mperham]

If you have errors that you expect to happen frequently enough to warrant this customization, it should be part of your app logic. Sidekiq's retry subsystem is for unexpected errors which are piped to your error service.

[helderasmoreira]

Fair enough. Thank you for confirming I wasn't missing anything in the current retry workflow.