chunked multi-part web posts #285
Comments
|
Yes, it is supported. |
|
here's more. It continues but will the same pattern |
|
Is there any update on this issue? I am experiencing the same type of error for my recorded future integration project. |
|
So your request data is: POST /cts/yeti_threat_service HTTP/1.1\r\n
Transfer-Encoding: chunked\r\n
Content-Type: multipart/form-data; boundary=ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC\r\n
Host: 192.168.1.215:9001\r\n
Connection: Keep-Alive\r\n
User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_181)\r\n
Accept-Encoding: gzip,deflate\r\n\r\n
f3\r\n
--ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC\r\n
Content-Disposition: form-data; name="artifact"\r\n
Content-Type: application/json; charset=UTF-8\r\n
Content-Transfer-Encoding: 8bit\r\n\r\n
{"type":"file.content","value":"cjohnsen_2020_12_21_attch_1_of_1.eml"}\r\nI reconstruced the chunk which is not yet send by your client and parsed the whole request with httoop: >>> import httoop
>>> x = httoop.ServerStateMachine('http', 'localhost', '80')
>>> d = b'POST /cts/yeti_threat_service HTTP/1.1\r\nTransfer-Encoding: chunked\r\nContent-Type: multipart/form-data; boundary=ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC\r\nHost: 192.168.1.215:9001\r\nConnection: Keep-Alive\r\nUser-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_181)\r\nAccept-Encoding: gzip,deflate\r\n\r\nf3\r\n--ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC\r\nContent-Disposition: form-data; name="artifact"\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n{"type":"file.content","value":"cjohnsen_2020_12_21_attch_1_of_1.eml"}\r\n'
>>> x.parse(d)
>>> x.state
{'startline': True, 'protocol': True, 'headers': True, 'body': False, 'trailer': False, 'method': True, 'uri': True}
>>> x.buffer
bytearray(b'')
>>> m, _ = x.parse(b'2e\r\n\r\n--ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC--\r\n\r\n0\r\n\r\n')[0]
>>> str(m.body)
'--ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC\r\nContent-Disposition: form-data; name="artifact"\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n{"type":"file.content","value":"cjohnsen_2020_12_21_attch_1_of_1.eml"}\r\n--ygdpVhgNZvCeNoHQdIOr2RvW7y3XIn6kYeIdXC--\r\n'
>>> z = m.body.decode()
>>> z[0].headers
<HTTP Headers([('Content-Disposition', b'form-data; name="artifact"'), ('Content-Type', b'application/json; charset=UTF-8'), ('Content-Transfer-Encoding', b'8bit')])>
>>> str(z[0])
'{"type":"file.content","value":"cjohnsen_2020_12_21_attch_1_of_1.eml"}'So your request seems to be valid and something, probably Would you share your circuits server code? I can reproduce something like this in httoop if I just parse the multipart body too early: >>> x = httoop.ServerStateMachine('http', 'localhost', '80')
>>> x.parse(d)
>>> x.message.body.decode()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "httoop/messages/body.py", line 142, in decode
self.data = codec.decode(self.__content_bytes(), self.encoding, self.mimetype)
File "httoop/codecs/multipart/multipart.py", line 31, in decode
raise DecodeError(_(u'Invalid multipart end: %r'), part.decode('ISO8859-1'))
httoop.exceptions.DecodeError: Invalid multipart end: '\r\nContent-Disposition: form-data; name="artifact"\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n{"type":"file.content","value":"cjohnsen_2020_12_21_attch_1_of_1.eml"}' |
|
https://github.com/circuits/circuits/blob/master/circuits/web/dispatchers/dispatcher.py#L111 this line is reached too early. |
|
I suspect this line is the cause: https://github.com/circuits/circuits/blob/master/circuits/web/http.py#L283 |
|
This is great news. Is there an expectation of when the change will be made and integrated into a release? |
|
@mscherfling probably not before in 3 weeks as I am in holiday. If I have too much time I can spend an evening maybe. |
|
@spaceone was a change which we can test in our environment? |
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
…ge body is not yet received completely
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
…ge body is not yet received completely
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
|
@mscherfling this is now fixed in the latest git master version. |
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
spaceone
added a commit
that referenced
this issue
Aug 20, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Does circuits supported chunked data decoding a web POST? Below are the headers and the stack trace when data is received with the header: Transfer-Encoding: chunked
Environment and Platform
The text was updated successfully, but these errors were encountered: