-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BugReport] Memory Leak on multipart_reader::currentHeaders #113
Comments
谢谢你提交issue,我会关注这个问题。 |
可以在feed()函数第一句输出buffer地址来确认这个问题,可以发现对于单次post请求,确实会有概率多次触发feed()函数。 |
用postman怎么做这个测试的? |
sorry我没有用postman,就在用firefox不断上传 27,046Bytes的图 |
测试我觉得可以模拟网络丢包试下,也许可以看到分片了。。 server端我是在feed()输出下第一个包的大小,如果第一个包差不多能把header分开,就在断点停一下。 |
Hello, I submit a PR, could you have interesting to merge it after testing?... |
你能准备一个测试的页面或者程序发出来吗,我想先用你提供的测试程序看看问题,确定问题在哪儿,然后再看pr |
build cinatra_example and run this code with python 3: import socket
import time
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 8080))
print('*Connected')
data1 = b'POST /upload_multipart HTTP/1.1\r\n' \
b'Host: 127.0.0.1\r\n' \
b'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0\r\n' \
b'Accept: */*\r\n' \
b'Accept-Language: en-US,en;q=0.5\r\n' \
b'Accept-Encoding: gzip, deflate\r\n' \
b'Content-Type: multipart/form-data; boundary=---------------------------191691572411478\r\n' \
b'Content-Length: 925\r\n' \
b'Connection: keep-alive\r\n' \
b'Pragma: no-cache\r\n' \
b'Cache-Control: no-cache\r\n' \
b'\r\n' \
b'-----------------------------191691572411478\r\n' \
b'Content-Disposition: form-data; name="na'
data2 = b'me"\r\n' \
b'\r\n' \
b'plaintext_document_withlongname.txt\r\n' \
b'-----------------------------191691572411478\r\n' \
b'Content-Disposition: form-data; name="type"\r\n' \
b'\r\n' \
b'text/plain\r\n' \
b'-----------------------------191691572411478\r\n' \
b'Content-Disposition: form-data; name="size"\r\n' \
b'\r\n' \
b'259\r\n' \
b'-----------------------------191691572411478\r\n' \
b'Content-Disposition: form-data; name="lastModifiedDate"\r\n' \
b'\r\n' \
b'undefined\r\n' \
b'-----------------------------191691572411478\r\n' \
b'Content-Disposition: form-data; name="file"; filename="plaintext_docu'
data3 = b'ment_withlongname.txt"\r\n' \
b'Content-Type: text/plain\r\n' \
b'\r\n' \
b'The 2011 European Women Basketball Championship, commonly called EuroBasket Women 2011, was the 33rd regional championship held by FIBA Europe. The competition was held in Poland from 2011. This was the 4th time that the EuroBasket Women was hosted by Poland.\r\n' \
b'-----------------------------191691572411478--\r\n'
print (data1 + data2 + data3)
s.send(data1)
print('*SENT -A')
time.sleep(1)
s.send(data2)
print('*SENT -B')
time.sleep(1)
s.send(data3)
print('*SENT -C')
time.sleep(1)
s.close()
print('*Close') |
在上传小文件时会偶尔翻车,加AddressSanitizer提示
function multipart_reader::cbHeaderEnd()
函数内heap-use-after-free on address。如图所示,将HTTP POST请求拆成2部分送达feed()函数
图链接
备用地址:draw.io
我们称第一个request fragment送来的地址是bufferA
第一次把bufferA送入
multipart_parser::feed()
后,状态机调用multipart_reader::cbHeaderEnd()
函数,将两个string_view存入multipart_headers::currentHeaders
中,结束后currentHeaders.size()==1
。第二个request fragment随之到来,存于bufferB
依旧是feed()调用cbHeaderEnd(),
此时bufferA已经释放,导致上述两个
multipart_headers::currentHeaders
中的string_view悬空程序再次调用
multipart_reader::cbHeaderEnd()
时,multimap试图读取原有string_views内容时读到了已经被释放的地址,在此处crash。如果幸运的话,分片不会把某一组header隔开,就不会触发bug。但也可以构造一个超级长的header,然后强行分片送抵服务器,就一定会触发。
解决方案?
想到把
using multipart_headers = std::multimap<std::string_view, std::string_view>;
改成
using multipart_headers = std::multimap<std::string, std::string>;
(我看到已经写了注释??)
cinatra/include/cinatra/multipart_reader.hpp
Lines 97 to 98 in ba17d4a
或者设计一个垃圾回收,触发
multipart_reader::cbHeadersEnd()
时,把之前的buffer再收掉。还有一个想法 buffer改用类似链表结构,把已经submit的string_view直接砍掉,会不会造成内存很零散?。。。
另外还有一个enhancement
在class upload_file里面,打开文件失败了请给个提示。。
第二组bug: feed()状态机不能处理多次送达的buffer
针对这几行举例,依然是两个post碎片,CR没有随着第一次feed(buffer1)到来,而在下一次feed(buffer2)时才送来。但是string_view引用的buffer1已经没了。
cinatra/include/cinatra/multipart_parser.hpp
Lines 148 to 161 in ba17d4a
第一次送来的buffer:
第二次feed送来的buffer:
错误依旧为 heap-use-after-free in cinatra::multipart_reader::cbHeaderEnd
解决方案?
同上吧,我觉得存下来比较好,我们没办法保证数据包在哪里断掉。。
第三组bug:当feed()分片到达时,可能会打断正在解析的header内容
在multipart_reader.hpp中
self->currentHeaderName
和self->currentHeaderValue
两个变量,第二次到达的会覆盖前一次到达的值。同样我觉得用string比较好操作,直接连接起来。。或者就滚动buffer,改下状态机不做multipart_parser.hpp#L192-L194
这个bug的确定理由:
The text was updated successfully, but these errors were encountered: