Python3 ThreadingHTTPServer fails to send chunked encoded response

[manuelb]

BPO	45523
Nosy	@vadmium
Files	myserver.py: source code reproducing the issue

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-19.23:00:07.416>
created_at = <Date 2021-10-19.13:42:01.112>
labels = ['type-bug', 'invalid', '3.9']
title = 'Python3 ThreadingHTTPServer fails to send chunked encoded response'
updated_at = <Date 2021-10-19.23:00:07.411>
user = 'https://bugs.python.org/manuelb'

bugs.python.org fields:

activity = <Date 2021-10-19.23:00:07.411>
actor = 'martin.panter'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-19.23:00:07.416>
closer = 'martin.panter'
components = []
creation = <Date 2021-10-19.13:42:01.112>
creator = 'manuel_b'
dependencies = []
files = ['50369']
hgrepos = []
issue_num = 45523
keywords = []
message_count = 2.0
messages = ['404300', '404373']
nosy_count = 2.0
nosy_names = ['martin.panter', 'manuel_b']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45523'
versions = ['Python 3.9']

[manuelb]

I'm implementing an HTTPServer class that produces a response with transfer-encoding chunked mode.

I'm sending the chunks as described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#chunked_encoding

If I send chunks of length <= 9 bytes the message is received correctly by the client, otherwise when sending chunks of length >= 10 bytes, it seems that some of them are not received and the message remains stuck in the client waiting indefinitely

In attachment an example of the complete code to reproduce the issue, but in short, the following code:

# writing 5 chunks of length 10
for i in range(5):
    text = str(i+1) * 10  # concatenate 10 chars
    chunk = '{0:d}\r\n'.format(len(text)) + text + '\r\n'
    self.wfile.write(chunk.encode(encoding='utf-8'))

# writing close sequence
close_chunk = '0\r\n\r\n'
self.wfile.write(close_chunk.encode(encoding='utf-8'))

Produces:
10\r\n
1111111111\r\n
10\r\n
2222222222\r\n
10\r\n
3333333333\r\n
10\r\n
4444444444\r\n
10\r\n
5555555555\r\n
0\r\n
\r\n

In this case the client hangs several minutes without a response

But if I use length 9 or less instead for example with

text = str(i+1) * 9

the client receives the correct body and the communication ends correctly (in about 6ms)

The problem persists with both http.server.ThreadingHTTPServer and http.server.HTTPServer
I tried also passing the body as an hard-coded binary string

some version informations:
Python 3.9.2
HTTP Client used: Postman 8.10, curl, Chrome

Thanks a lot for any help
Manuel

[vadmium]

Looks like you forgot to encode the length of ten in hexadecimal.

I don't think the HTTP server module has any special handling for chunked responses, so this up to the user and isn't a bug in Python.