Error: Response payload is not completed

[zzc0430]

Describe the bug

When using the acreate interface and attempting to utilize streaming output, I encountered "aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed"

• Time: (UTC) 2023-08-17 06:08:24,030 ~ 2023-08-17 06:13:24,180
• Request ID: 9c46544d6517b7b8e9df1e9e61fd7f53

To Reproduce

1. using acreate interface
2. stream = True

Code snippets

async for chunk in completion_stream:
File "/usr/local/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in <genexpr>
return (
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 319, in wrap_resp
async for r in resp:
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 633, in <genexpr>
return (
File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 114, in parse_stream_async
async for line in rbody:
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 35, in __anext__
rv = await self.read_func()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 311, in readline
return await self.readuntil()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 343, in readuntil
await self._wait("readuntil")
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 304, in _wait
await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed

OS

Linux

Python version

Python v3.9

Library version

openai-python v0.27.2

[logankilpatrick]

cc @dereklegenzoff

[ShantanuNair]

Looks like this is a regression of a bug on openai servers: openai-php/laravel#36 and #399

[ShantanuNair]

@zzc0430 Did you fix this? I am running into the same issue on langchain using 3.5-turbo-16k where I have long input prompts of about 10-12k tokens, and see this error. I am not using streaming.

[zzc0430]

@zzc0430 Did you fix this? I am running into the same issue on langchain using 3.5-turbo-16k where I have long input prompts of about 10-12k tokens, and see this error. I am not using streaming.

@ShantanuNair Not yet, it looks like a server-side issue.

[ShantanuNair]

@zzc0430 What about this - aio-libs/aiohttp#4581 (comment) ?
I have a feeling this is potentially not the issue, but the issue is actually from session setting in the openai client.

I may have narrowed it down, but could use your help debugging - What version of openai do you have installed with langchain when you run into this issue? Try upgrading and see if it's fixed?

[zzc0430]

@zzc0430 What about this - aio-libs/aiohttp#4581 (comment) ? I have a feeling this is potentially not the issue, but the issue is actually from session setting in the openai client.

I may have narrowed it down, but could use your help debugging - What version of openai do you have installed with langchain when you run into this issue? Try upgrading and see if it's fixed?

@ShantanuNair The version I'm using is 0.27.2, and I will try to update to 0.28.0, but it seems that this bug cannot be consistently reproduced.

[ShantanuNair]

@zzc0430 Let's do our best to find a root cause, if you do figure out a way to reproduce it let's work on it. I am trying to make sure I never see this again. Do you also see 502 errors Cloudflare bad_gateway much? My environment is modal.com Please let me know if this is at all similar for you, we may be able to narrow it down.

[zzc0430]

@zzc0430 Let's do our best to find a root cause, if you do figure out a way to reproduce it let's work on it. I am trying to make sure I never see this again. Do you also see 502 errors Cloudflare bad_gateway much? My environment is modal.com Please let me know if this is at all similar for you, we may be able to narrow it down.

Similarly, I've also encountered 502 errors.

[ShantanuNair]

@zzc0430 Appreciate it!! I might be able to narrow this down. Can you provide a few details so I can check what we have in common?

• What environment are you running chains on? Local like macOS? Some compute provider like AWS Lambda/EC2 or Modal like in my case?
• Do you have the default retries (max_retries default is 6) or retries set for your chain? In that case, does your retry complete after a 502? In my case all further retries for that chain fails until my compute hits its timeout.
• Does the above mentioned retry run after the time it's supposed to? Mine says retry in 4/8s etc but ends up hanging for 6-7 minutes before running the retry! This could be related to your OP issue. I'm investigating potential causes in langchain lib and the openai-python client.

Really appreciate if you could help me out with these details.

[ShantanuNair]

Getting this on the latest openai-python version

[zzc0430]

Getting this on the latest openai-python version

@ShantanuNair Thanks, which commit fixed this issue?

[ShantanuNair]

@zzc0430 I meant to say it is still occurring on the latest version.