Override HTTP Status Codes that are Retried

[debugmiller]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

We use the splunk_hec sink to send data to Splunk Cloud. Unfortunately Splunk Cloud occasionally returns HTTP 404 which, understandably, causes Vector to drop data with the following message: Not retriable; dropping the request. The situation is always transient, lasting only a few seconds.

Attempted Solutions

We have talked extensively with Splunk support and it seems like this is known (expected) behavior and happens intermittently when they are doing backend maintenance. They do not expect to fix this behavior.

Proposal

It would be great if the Splunk HEC sink (and presumably HTTP if thats the underlying sink) allowed us to override the HTTP status codes that are retried so that we could add 404 to the list of status codes that are retried. This would let us prevent Vector from dropping data.

References

No response

Version

vector 0.22.0 (x86_64-apple-darwin 5e937e3 2022-06-01)

[jszwedko]

Hi @debugmiller !

Thanks for opening this. I think this is related to #10870 so I'll close this and link it there, but feel free to add any more details to that issue. I think this is something we will be picking up in Q3.