Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tengine上传大文件支持问题 #1003

Open
xianshi122 opened this issue Jan 29, 2018 · 16 comments
Open

tengine上传大文件支持问题 #1003

xianshi122 opened this issue Jan 29, 2018 · 16 comments

Comments

@xianshi122
Copy link

@xianshi122 xianshi122 commented Jan 29, 2018

从nginx1.10.1升级为tengin2.1.2个后,
用户通过multipart/form-data 上传文件时
如果文件大于1M就会上传失败
抓包分析后,发现在用户在上传大文件时,http body内容不完整
导致 java MultipartHttpServletRequest request 拿到的 request.getFileNames() size 为0

抓取了 tengin与tomcat之间的报文,异常报文中 http body内容异常
正常报文
`
POST /attachment/upload HTTP/1.0
Host: .com
X-Real-IP: 10.1.1.15
X-Forwarded-For: 10.1.1.15
Connection: close
Content-Length: 396752
accept: application/json, text/javascript, /; q=0.01
origin: https://
.com
x-requested-with: XMLHttpRequest
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
content-type: multipart/form-data; boundary=----WebKitFormBoundaryn1FCXAuaQn8567yc
referer: https://qa-manage.******/104407
accept-encoding: gzip, deflate, br
accept-language: zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7
cookie: gr_user_id=6f4a05ba-9acc-438e-b904-fc580cfdf390; ***=2f854935-21d9-4f9e-90c8-cfbcd9d1c735; _ga=GA1.2.699820353.1517203602; _gid=GA1.2.485202220.1517203602; Hm_lvt_5d14949ada92c16ee007021e4d7234db=1516938388,1517073337,1517195006,1517202090; Hm_lpvt_5d14949ada92c16ee007021e4d7234db=1517208531; ***=48fd5991-f24d-424a-9f54-4245d28bfb6f; langmanage=en_US

------WebKitFormBoundaryn1FCXAuaQn8567yc
Content-Disposition: form-data; name="saveToUserKey"

false
------WebKitFormBoundaryn1FCXAuaQn8567yc
Content-Disposition: form-data; name="userAttachOnly"

undefined
------WebKitFormBoundaryn1FCXAuaQn8567yc
Content-Disposition: form-data; name="titles"

..............................
------WebKitFormBoundaryn1FCXAuaQn8567yc
Content-Disposition: form-data; name="files[]"; filename="...............................pptx"
Content-Type: application/vnd.openxmlformats-officedocument.presentationml.presentation
.....more
异常报文POST /attachment/upload HTTP/1.0
Host: qa-**.com
X-Real-IP: 10.1.1.15
X-Forwarded-For: 10.1.1.15
Connection: close
Content-Length: 5342347
accept: application/json, text/javascript, /; q=0.01
origin: https://
.com
x-requested-with: XMLHttpRequest
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
content-type: multipart/form-data; boundary=----WebKitFormBoundaryWM42hdx6CVJtNPxz
referer: https://qa-***/detail/104407
accept-encoding: gzip, deflate, br
accept-language: zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7
cookie: gr_user_id=6f4a05ba-9acc-438e-b904-fc580cfdf390; *****=2f854935-21d9-4f9e-90c8-cfbcd9d1c735; _ga=GA1.2.699820353.1517203602; _gid=GA1.2.485202220.1517203602; Hm_lvt_5d14949ada92c16ee007021e4d7234db=1516938388,1517073337,1517195006,1517202090; Hm_lpvt_5d14949ada92c16ee007021e4d7234db=1517208531; *****=48fd5991-f24d-424a-9f54-4245d28bfb6f; langmanage=en_US

0.................DN.ppt/media/image28.pngPK..-.
.......!...NaP=..P=................fuN.ppt/media/image32.pngPK..-.
.......!._F^QJ=..J=..................N.ppt/media/image31.pngPK..-.
.......!.e6V..:...:................f.N.ppt/media/image30.pngPK..-.
.......!."
*..6...6................F+O.ppt/media/image29.pngPK..-.
.......!..) .Ne..Ne.................aO.ppt/media/image27.pngPK..-.
.......!....h.4...4................y.O.ppt/media/image33.pngPK..-.
.......!...juuG..uG..................O.ppt/media/image35.pngPK..-.
.......!..&0.q<..q<................iCP.ppt/media/image38.pngPK..-.
.......!.....b1..b1................ more
`

@taoyuanyuan
Copy link
Contributor

@taoyuanyuan taoyuanyuan commented Jan 30, 2018

错误日志有啥信息吗? 麻烦贴一下配置文件

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 6, 2018

类似的问题:
版本最新的2.2.2 centos 6.7 64位
只有在https+http2 启用后,上传1MB以上的文件,就会卡住,后端java收到为空

验证:

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http2 -vvv https://x.xx.com/api/rent/productImage/uploadImage    --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x231bf90)
> POST /api/rent/productImage/uploadImage HTTP/2
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------7d5aff34511dd99a
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!

会在这里卡住,死掉

如果换用http1.1 请求一点问题没有

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http1.1 -vvv https://x.x.com/api/rent/productImage/uploadImage   --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
> POST /api/rent/productImage/uploadImage HTTP/1.1
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------6e340c4b050537bb
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 
< Server: ch999ws19
< Date: Tue, 06 Mar 2018 08:34:42 GMT
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Application-Context: rent:prod:9987
< Set-Cookie: buyCartCookieId=970940723212963842; Max-Age=86400; Expires=Wed, 07-Mar-2018 08:34:42 GMT; Path=/
< 
* Connection #0 to host x.x.com left intact
{"code":0,"msg":"SUCCESS","userMsg":"上传成功!","data":"https://x.x.com/newstatic/794/3a6fc89460638a.png","area":{}}

配置文件:



server {

       server_name  zu.xxx.com;

listen  443 ssl http2;
ssl_certificate conf.d/cerNew/xxx_com.crt;
ssl_certificate_key conf.d/cerNew/xxx_com.key;

    root /var/www/xxx/;
    index index.html;

location /api {
           proxy_pass http://zuji;
           proxy_set_header X-Real-IP       $remote_addr;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_set_header XHttpScheme "$scheme";
	       #proxy_buffering off;
        }

nginx.conf

http {
    ssl_session_cache    shared:SSL:1000m;
    ssl_session_timeout  10m;
     
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_prefer_server_ciphers   on;

	client_max_body_size	1024m;

nginx 模块

[root@linux2 ~]# /usr/local/nginx/sbin/nginx -V
Tengine version: Tengine/2.2.2 (nginx/1.8.1)
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_gzip_static_module --with-http_concat_module --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2n --with-http_lua_module --with-luajit-lib=/usr/local/lib/ --with-luajit-inc=/usr/local/include/luajit-2.0/ --with-lua-inc=/usr/local/include/luajit-2.0/ --with-lua-lib=/usr/local/lib/ --with-ld-opt=-Wl,-rpath,
nginx: loaded modules:
nginx:     ngx_core_module (static)
nginx:     ngx_errlog_module (static)
nginx:     ngx_conf_module (static)
nginx:     ngx_dso_module (static)
nginx:     ngx_events_module (static)
nginx:     ngx_event_core_module (static)
nginx:     ngx_epoll_module (static)
nginx:     ngx_procs_module (static)
nginx:     ngx_proc_core_module (static)
nginx:     ngx_openssl_module (static)
nginx:     ngx_regex_module (static)
nginx:     ngx_http_module (static)
nginx:     ngx_http_core_module (static)
nginx:     ngx_http_log_module (static)
nginx:     ngx_http_upstream_module (static)
nginx:     ngx_http_v2_module (static)
nginx:     ngx_http_static_module (static)
nginx:     ngx_http_gzip_static_module (static)
nginx:     ngx_http_autoindex_module (static)
nginx:     ngx_http_index_module (static)
nginx:     ngx_http_concat_module (static)
nginx:     ngx_http_auth_request_module (static)
nginx:     ngx_http_auth_basic_module (static)
nginx:     ngx_http_access_module (static)
nginx:     ngx_http_limit_conn_module (static)
nginx:     ngx_http_limit_req_module (static)
nginx:     ngx_http_geo_module (static)
nginx:     ngx_http_map_module (static)
nginx:     ngx_http_split_clients_module (static)
nginx:     ngx_http_referer_module (static)
nginx:     ngx_http_rewrite_module (static)
nginx:     ngx_http_ssl_module (static)
nginx:     ngx_http_proxy_module (static)
nginx:     ngx_http_fastcgi_module (static)
nginx:     ngx_http_uwsgi_module (static)
nginx:     ngx_http_scgi_module (static)
nginx:     ngx_http_memcached_module (static)
nginx:     ngx_http_empty_gif_module (static)
nginx:     ngx_http_browser_module (static)
nginx:     ngx_http_user_agent_module (static)
nginx:     ngx_http_upstream_hash_module (static)
nginx:     ngx_http_upstream_ip_hash_module (static)
nginx:     ngx_http_upstream_consistent_hash_module (static)
nginx:     ngx_http_upstream_check_module (static)
nginx:     ngx_http_upstream_least_conn_module (static)
nginx:     ngx_http_upstream_keepalive_module (static)
nginx:     ngx_http_upstream_dynamic_module (static)
nginx:     ngx_http_stub_status_module (static)
nginx:     ngx_http_write_filter_module (static)
nginx:     ngx_http_header_filter_module (static)
nginx:     ngx_http_chunked_filter_module (static)
nginx:     ngx_http_v2_filter_module (static)
nginx:     ngx_http_range_header_filter_module (static)
nginx:     ngx_http_gzip_filter_module (static)
nginx:     ngx_http_postpone_filter_module (static)
nginx:     ngx_http_ssi_filter_module (static)
nginx:     ngx_http_charset_filter_module (static)
nginx:     ngx_http_userid_filter_module (static)
nginx:     ngx_http_footer_filter_module (static)
nginx:     ngx_http_trim_filter_module (static)
nginx:     ngx_http_headers_filter_module (static)
nginx:     ngx_http_upstream_session_sticky_module (static)
nginx:     ngx_http_reqstat_module (static)
nginx:     ngx_http_lua_module (static)
nginx:     ngx_http_copy_filter_module (static)
nginx:     ngx_http_range_body_filter_module (static)
nginx:     ngx_http_not_modified_filter_module (static)
[root@linux2 ~]# 

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 6, 2018

上述同样的配置,在2.2.0 版本没问题

[root@linux2 sbin]# /usr/local/nginx/sbin/nginx -V
Tengine version: Tengine/2.2.0 (nginx/1.8.1)
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_gzip_static_module --with-http_concat_module --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2j --add-module=../ngx_cache_purge-2.3 --with-http_lua_module --with-luajit-lib=/usr/local/lib/ --with-luajit-inc=/usr/local/include/luajit-2.0/ --with-lua-inc=/usr/local/include/luajit-2.0/ --with-lua-lib=/usr/local/lib/ --with-ld-opt=-Wl,-rpath,
nginx: loaded modules:
nginx:     ngx_core_module (static)
nginx:     ngx_errlog_module (static)
nginx:     ngx_conf_module (static)
nginx:     ngx_dso_module (static)
nginx:     ngx_events_module (static)
nginx:     ngx_event_core_module (static)
nginx:     ngx_epoll_module (static)
nginx:     ngx_procs_module (static)
nginx:     ngx_proc_core_module (static)
nginx:     ngx_openssl_module (static)
nginx:     ngx_regex_module (static)
nginx:     ngx_http_module (static)
nginx:     ngx_http_core_module (static)
nginx:     ngx_http_log_module (static)
nginx:     ngx_http_upstream_module (static)
nginx:     ngx_http_v2_module (static)
nginx:     ngx_http_static_module (static)
nginx:     ngx_http_gzip_static_module (static)
nginx:     ngx_http_autoindex_module (static)
nginx:     ngx_http_index_module (static)
nginx:     ngx_http_concat_module (static)
nginx:     ngx_http_auth_request_module (static)
nginx:     ngx_http_auth_basic_module (static)
nginx:     ngx_http_access_module (static)
nginx:     ngx_http_limit_conn_module (static)
nginx:     ngx_http_limit_req_module (static)
nginx:     ngx_http_geo_module (static)
nginx:     ngx_http_map_module (static)
nginx:     ngx_http_split_clients_module (static)
nginx:     ngx_http_referer_module (static)
nginx:     ngx_http_rewrite_module (static)
nginx:     ngx_http_ssl_module (static)
nginx:     ngx_http_proxy_module (static)
nginx:     ngx_http_fastcgi_module (static)
nginx:     ngx_http_uwsgi_module (static)
nginx:     ngx_http_scgi_module (static)
nginx:     ngx_http_memcached_module (static)
nginx:     ngx_http_empty_gif_module (static)
nginx:     ngx_http_browser_module (static)
nginx:     ngx_http_user_agent_module (static)
nginx:     ngx_http_upstream_hash_module (static)
nginx:     ngx_http_upstream_ip_hash_module (static)
nginx:     ngx_http_upstream_consistent_hash_module (static)
nginx:     ngx_http_upstream_check_module (static)
nginx:     ngx_http_upstream_least_conn_module (static)
nginx:     ngx_http_upstream_keepalive_module (static)
nginx:     ngx_http_upstream_dynamic_module (static)
nginx:     ngx_http_stub_status_module (static)
nginx:     ngx_http_cache_purge_module (static)
nginx:     ngx_http_write_filter_module (static)
nginx:     ngx_http_header_filter_module (static)
nginx:     ngx_http_chunked_filter_module (static)
nginx:     ngx_http_v2_filter_module (static)
nginx:     ngx_http_range_header_filter_module (static)
nginx:     ngx_http_gzip_filter_module (static)
nginx:     ngx_http_postpone_filter_module (static)
nginx:     ngx_http_ssi_filter_module (static)
nginx:     ngx_http_charset_filter_module (static)
nginx:     ngx_http_userid_filter_module (static)
nginx:     ngx_http_footer_filter_module (static)
nginx:     ngx_http_trim_filter_module (static)
nginx:     ngx_http_headers_filter_module (static)
nginx:     ngx_http_upstream_session_sticky_module (static)
nginx:     ngx_http_reqstat_module (static)
nginx:     ngx_http_lua_module (static)
nginx:     ngx_http_copy_filter_module (static)
nginx:     ngx_http_range_body_filter_module (static)
nginx:     ngx_http_not_modified_filter_module (static)
[root@linux2 sbin]# 

请求记录:

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http2 -vvv https://x.x.com/api/rent/productImage/uploadImage    --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x100df90)
> POST /api/rent/productImage/uploadImage HTTP/2
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------355178d905bb1f52
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< server: nginx
< date: Tue, 06 Mar 2018 08:49:38 GMT
< content-type: application/json;charset=UTF-8
< x-application-context: rent:prod:9987
< set-cookie: buyCartCookieId=970944482521501697; Max-Age=86400; Expires=Wed, 07-Mar-2018 08:49:38 GMT; Path=/
< 
* Connection #0 to host x.x.com left intact
{"code":0,"msg":"SUCCESS","userMsg":"上传成功!","data":"https://x.x.com/newstatic/797/3a719e59ef8a69.png","area":{}}
@yourchanges
Copy link

@yourchanges yourchanges commented Mar 6, 2018

不用curl 用最新的chrome 64 也是一样的
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 6, 2018

有没有任何提示,代码在相关代码在哪个模块,我好自己去看了尝试修复,生产环境,急

@chobits
Copy link
Member

@chobits chobits commented Mar 7, 2018

proxy_request_buffering这个指令你有配置吗?这个指令在2.1.2里要设置成off才可以,因为这个版本基于的ngx还不支持http2的流式上传

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 7, 2018

非常感谢,的确目前的2.1.2 和2.2.2 里面的http2 还不支持stream下的请求缓存,设置 proxy_request_buffering off;
即可解决问题

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 7, 2018

那么小于1MB的文件,怎么就可以正常上传?是否我调大buffer大小(如果有这个参数,可以调整的话)也可以避免这个问题。
另外,http2的上传支持实现,有计划表了么?

@chobits
Copy link
Member

@chobits chobits commented Mar 7, 2018

  1. tengine-2.2.2里的支持流式上传
  2. tengine-2.1.2里,上传数据过大(超过client_body_buffer_size)会写磁盘的,不超过就放在内存里。所以对于上传来说可用性没啥问题,但是会把完整上传数据收完才发给后端java

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 7, 2018

@chobits
Copy link
Member

@chobits chobits commented Mar 7, 2018

那就有bug 我用的2.2.2有问题 2.2.0没问题

你这里都是设置 'proxy_request_buffering off;'测试的吗?

这周我会看下,因为h2还有一些代码更新下没merge进来

@yourchanges
Copy link

@yourchanges yourchanges commented Mar 8, 2018

@chobits 是的,在location 段加的proxy_request_buffering off;

@lvai125
Copy link

@lvai125 lvai125 commented May 31, 2018

                    client_header_timeout 3600;
                    keepalive_timeout     3600;
                    client_body_timeout 3600;
                    send_timeout          3600;
                    client_max_body_size 200m;  
                    proxy_request_buffering off;
                    proxy_connect_timeout   3600; 
                    proxy_read_timeout 3600;
                    proxy_send_timeout 3600;
                    client_header_buffer_size 12800k;
                    large_client_header_buffers 4 12800k;
                    proxy_ignore_client_abort on;
                    proxy_temp_file_write_size 12800k;
@lvai125
Copy link

@lvai125 lvai125 commented May 31, 2018

上面是我的配置,在http里上传图片没问题1m左右。而在https里上传超过10k就报400。加了proxy_request_buffering off;这个参数解决了。

@chobits chobits changed the title tenginx上传大文件支持问题 tengine上传大文件支持问题 Jun 14, 2018
@lazyfighter
Copy link

@lazyfighter lazyfighter commented Jan 23, 2019

这个已经修复了吗

@chobits
Copy link
Member

@chobits chobits commented Mar 14, 2019

Hi all

Please try our latest master branch, we have merged all HTTP/2 bugfix.

Hi all, we have merged nginx 1.15.9 into latest master. Please check it! We'll release new version 2.3.x (based on nginx 1.15.x) as soon. Please check WARNING list before upgrade. If have any questions, please give us feedback in time. Thank you.

WARNING

Tengine has been upgraded core files from the Nginx 1.15.9 version. Note that this version is slightly not backwards compatible, some tengine features has been replaced by nginx offical, check the following list:

  1. Removed reuse_port directive implemented by Tengine and use the official reuseport of Nginx, detailed reference document .
  2. Removed dso_tool tool and dso directive, If before using Tengine dso feature, you could switch to the Nginx official load_module directive, detailed reference document1document2 .
  3. Moved the slice module into modules dir, use the Nginx official slice feature by default and use --add-module=modules/ngx_http_slice_module way to compile if need Tengine's slice module, otherwise use --with-http_slice_module compilation parameter.
  4. Extract the tengine modules to modules dir, if you need to use that module then use --add-module=modules/<module_name> way to compile.
  5. The request counting logic for limit_req is consistent with the Nginx official, removed the logic that skip counting with any empty value variable in limit_req_zone.

各位Tengine使用者你们好!我们近期合并了Nginx官方1.15.9版本代码到Tengine master分支,大家可以先尝鲜使用master分支代码做测试(若上生产使用请一定要做好灰度验证、并阅读相关注意列表)。同时我们最近也会正式发布Tengine-2.3.x版本,欢迎大家使用,如有任何问题请随时反馈,谢谢。

警告

本次Tengine升级core代码到官方Nginx 1.15.9版本,由于其部分功能Nginx官方当前已经具备、所以本次直接弃用Tengine自身实现的部分配置指令,具体不兼容列表如下:

  1. 废弃Tengine自身实现的reuse_port指令,使用Nginx官方的reuseport。升级方法:将events配置块里面的reuse_port on|off注释掉,在对应的监听端口后面加reuseport参数、详细参考文档
  2. 废弃Tengine的dso_tool工具以及dso配置指令,若之前有使用Tengine的dso功能、则可以切换到Nginx官方的load_module指令,详细文档参考1参考2
  3. 移除Tengine加强版slice模块到modules、默认使用Nginx官方slice功能,如果依然需要使用Tengine的slice则编译slice时请使用 --add-module=modules/ngx_http_slice_module,否则使用 --with-http_slice_module 编译参数
  4. Tengine自身实现的模块,当前全部剥离到modules目录下,如果需要使用那个模块、则使用--add-module=modules/<module_name>的方式进行编译
  5. limit_req的请求计数逻辑和官方保持一致,去除limit_req_zone中任何一个变量值为空跳过请求计数的逻辑
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants