Write to InfluxDB cloud fails with ApiException: (503) Reason: Service Unavailable; upstream connect error or disconnect/reset before headers. reset reason: connection failure #326
Description
Steps to reproduce:
Using the Python client make repeated calls to the write API.
Do this simultaneously from 2 or 3 different clients (not sure if this is relevant but this is the fail condition)
Write rates are a call to the API every second or so
I'm using the SYNCHRONOUS calls
influx_client = InfluxDBClient(url=os.environ['influx_url'], token=os.environ['token'],retries=retries) write_api = influx_client.write_api(write_options=SYNCHRONOUS)
(repeatedly)
influx_returns = write_api.write(InfluxBucket, my_org, Influx_lines,'ms')
Expected behavior:
Until 17th Aug we were seeing the expected behaviour.
All the writes succeeded.
Data was written reliably
Actual behavior:
From 17th August we started getting
ApiException: (503)
Reason: Service Unavailable; upstream connect error or disconnect/reset before headers.
reset reason: connection failure
The errors definitely started on 17th Aug and I know we didn't change anything because everyone was on holiday!
I have AWS logs which show when this started
You can see the discussion here that it's affecting multiple users now
Specifications:
- Python 3.8:
- InfluxDB Version: Cloud v2
- Platform: AWS Lambda written in Python