Intermittent errors caused by Kafka Flush Timeout #75
Description
Describe the bug
The message publish to Kafka fails intermittently with this error:
Caused by org.apache.http.client.HttpResponseException: status code: 500, reason phrase: POST request to https://event-gate-api-prod.pdintdebdtools.aws.dsarena.com/topics/public.cps.za.runs was unsuccessful. Status code: 500, response body: Some({"success": false, "statusCode": 500, "errors": [{"type": "kafka", "message": "Kafka flush timeout after 5.0s: 1 message(s) still pending"}]})
To Reproduce
Happens intermittently
Expected behavior
The connection to Kafka should handle the error. Either retry or produce only warning if rest of the ways to produce messages fail.
Screenshots
