EventSensor DLQ after retry exhaustion

[talebzeghmi]

Is your feature request related to a problem? Please describe.
We've hit two scenarios where EventSensor trigger failures result in messages being lost because they failed to trigger a WorkflowTemplate due to a webhook intermittent failure, or a deployment error where the WorkflowTemplate was not found.

Describe the solution you'd like
Just as there is a way to add to a dead letter queue (DLQ) in the EventSource for failure to process, specifying a DLQ in the EventSensor would be the final destination after all retries have exhausted and give confidence in the argo eventing system to not lose messages.

Describe alternatives you've considered
n/a

Additional context
I'm also considering adding trace (or correlationId) logging to EventSource and EventSensor s, see #2323 to easily find and correlate the message that failed to trigger with outside system logs.

see: https://cloud-native.slack.com/archives/C01TNKD6KL6/p1661631182229819

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.