[Task]: RunInference - send failures to dead letter queue

[damccorm]

What needs to happen?

Right now, if RunInference fails a batch inference, it fails the whole transform. For batch pipelines, this means failing the pipeline on non-retryable failures (which represent most inference failures), for streaming it means infinite retries and a stuck pipeline.

We should handle failures by passing them to the next step as part of the PredictionResult object instead so that users can perform custom error handling. We should also document this behavior in the PyDoc and on our website.

Issue Priority

Priority: 2

Issue Component

Component: run-inference

[damccorm]

@BjornPrime this is a good one to pick up if you have space in the future (if you finish your current work or are blocked)

[BjornPrime]

.take-issue

[AnandInguva]

@damccorm what is the action item on this one?

Should I pick this up as an effort to model updates? or we can add it once the model updates is out

[damccorm]

I would address it separately, but if you would like to do so it would be helpful

[AnandInguva]

.take-issue