Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Added resubmit handlers on Pulsar connection failures #7053
Currently, the Pulsar runner does not have support for resubmit handlers. This PR adds support for resubmit handlers to the Pulsar runner if an unknown error occurs during job preparation or finalisation. For example:
In addition, it also introduces a referrer parameter for dynamic destinations. The referrer parameter is useful for finding out the resubmit conditions of a dynamic destination's referrer, or any of its parameters.
With the referrer parameter being accessible within the dynamic function as:
I think the problem is finish being called there? I can see you getting a command_line in production but the transport failing so that you'd get a chance to call fail_job without finish. Regardless of the specifics, I feel like the approach I pushed is a bit more uniform and it seems to cause the now simplified test of resubmission to pass. Locally this test takes a minute or two to pass though - I'm a little concerned about including it in the suite of tests that run with each PR for that reason.