发现listener_wrappers参数影响反向代理gRPC应用。

[lxhao61]

发现 listener_wrappers参数影响反向代理gRPC应用，希望大神修复。
当caddy与Xray或v2ray配合，对Xray或v2ray的WebSocket、h2c、gRPC进行反向代理，且同时naiveproxy与trojian-go代理时（caddy插件），发现仅反向代理gRPC无法正常工作。情况是：
1、配置listener_wrappers参数时，在CDN模式下反向代理gRPC正常，如cloudflare的CDN；正常模式下反向代理gRPC反而不正常。
2、去除listener_wrappers参数时，在CDN模式下与正常模式下，反向代理gRPC都正常；但trojian-go代理无法用了。

相关应用见示例：
反向代理gRPC应用 ：V2Ray(SS+gRPC)+Nginx\Caddy
反向代理综合应用：V2Ray(B+D+G+A)+Caddy(N+T)

[imgk]

According to your description, trojan-go module is one of the reasons resulting in failure of gRPC. The other one is the client. Because when you use a CDN, everything works fine. Can you give more details about this phenomenon? For example, caddy logs.

[lxhao61]

According to your description, trojan-go module is one of the reasons resulting in failure of gRPC. The other one is the client, Because when you use a CDN, everything works fine. Can you give more details about this phenomenon? For example, caddy logs.

是的，具体表现就是listener_wrappers参数配置与去除。
1、配置listener_wrappers参数时，正常模式下反向代理gRPC，caddy日志如下：

{"level":"warn","ts":1627067602.2141023,"logger":"admin","msg":"admin endpoint disabled"}
{"level":"info","ts":1627067602.2178848,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0005f6150"}
{"level":"warn","ts":1627067602.2309215,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [cloudflare origin certificate *.x.z x.z]: no URL to issuing certificate"}
{"level":"info","ts":1627067602.2311654,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"x.z","server_name":"https"}
{"level":"info","ts":1627067602.2311907,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"*.x.z","server_name":"https"}
{"level":"info","ts":1627067602.2312036,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"https"}
{"level":"debug","ts":1627067603.8596509,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1627067603.8597584,"logger":"http","msg":"enabling experimental HTTP/3 listener","addr":":443"}
{"level":"debug","ts":1627067603.8598228,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":true,"tls":true}
{"level":"info","ts":1627067603.8598416,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["*.x.y","x.y"]}
{"level":"error","ts":1627067603.8613348,"msg":"unable to create folder for config autosave","dir":"caddy","error":"mkdir caddy: permission denied"}
{"level":"info","ts":1627067603.8666222,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/home/tls"}
{"level":"info","ts":1627067603.8681648,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1627067603.8695576,"msg":"serving initial configuration"}
{"level":"warn","ts":1627067603.869576,"msg":"$HOME environment variable is empty - please fix; some assets might be stored in ./caddy"}
{"level":"error","ts":1627067636.9175153,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067642.4182901,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067648.1762357,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067654.3751583,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067660.819949,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067668.1480823,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067676.2648427,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067689.2518802,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067708.9778237,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067739.708198,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067774.0379121,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}
{"level":"error","ts":1627067807.4069955,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}

2、其它情况，各个日志对比后发现雷同，故提供一个日志见如下：

{"level":"warn","ts":1627068102.0000696,"logger":"admin","msg":"admin endpoint disabled"}
{"level":"info","ts":1627068102.0022943,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0005f4150"}
{"level":"warn","ts":1627068102.0172973,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [cloudflare origin certificate *.x.z x.z]: no URL to issuing certificate"}
{"level":"info","ts":1627068102.0175743,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"x.z","server_name":"https"}
{"level":"info","ts":1627068102.0175996,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"*.x.z","server_name":"https"}
{"level":"info","ts":1627068102.0176098,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"https"}
{"level":"info","ts":1627068103.712377,"logger":"http","msg":"enabling experimental HTTP/3 listener","addr":":443"}
{"level":"debug","ts":1627068103.7125409,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":true,"tls":true}
{"level":"debug","ts":1627068103.712592,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1627068103.712607,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["x.y","*.x.y"]}
{"level":"error","ts":1627068103.7140703,"msg":"unable to create folder for config autosave","dir":"caddy","error":"mkdir caddy: permission denied"}
{"level":"info","ts":1627068103.719029,"msg":"serving initial configuration"}
{"level":"warn","ts":1627068103.7190502,"msg":"$HOME environment variable is empty - please fix; some assets might be stored in ./caddy"}
{"level":"info","ts":1627068103.7191465,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/home/tls"}
{"level":"info","ts":1627068103.72056,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"debug","ts":1627068137.4242604,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:13828","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ray":["673746d03e000560-LAX"],"X-Forwarded-Proto":["https"],"Cf-Ipcountry":["CN"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Te":["trailers"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"debug","ts":1627068137.435861,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:13826","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Cf-Ipcountry":["CN"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Cf-Ray":["673746d03e040560-LAX"],"Te":["trailers"],"X-Forwarded-Proto":["https"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"debug","ts":1627068137.7105577,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:14332","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cf-Connecting-Ip":["218.6.202.25"],"Cdn-Loop":["cloudflare"],"Cf-Ipcountry":["CN"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"X-Forwarded-Proto":["https"],"User-Agent":["grpc-go/1.36.1"],"Accept-Encoding":["gzip"],"Cf-Ray":["673746d2189f0560-LAX"],"Te":["trailers"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"debug","ts":1627068137.7328389,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:14364","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cf-Ipcountry":["CN"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ray":["673746d238d10560-LAX"],"Te":["trailers"],"X-Forwarded-Proto":["https"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068139.7976937,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"error","ts":1627068139.801879,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"error","ts":1627068146.8654711,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"debug","ts":1627068156.4674268,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:43328","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cdn-Loop":["cloudflare"],"Cf-Ipcountry":["CN"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Te":["trailers"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Accept-Encoding":["gzip"],"Cf-Ray":["673747474c160560-LAX"],"X-Forwarded-Proto":["https"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068156.869175,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"debug","ts":1627068166.4228206,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:58786","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"User-Agent":["grpc-go/1.36.1"],"Te":["trailers"],"X-Forwarded-Proto":["https"],"Content-Type":["application/grpc"],"Cf-Connecting-Ip":["218.6.202.25"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"Cf-Ray":["67374785685c0560-LAX"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068166.866175,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"debug","ts":1627068188.8323975,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:58786","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cf-Connecting-Ip":["218.6.202.25"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"Cf-Ray":["673748132b7b0560-LAX"],"Te":["trailers"],"X-Forwarded-Proto":["https"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"debug","ts":1627068191.4179814,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:39740","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"X-Forwarded-Proto":["https"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Cf-Ray":["6737481e8d8d0560-LAX"],"Te":["trailers"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Content-Type":["application/grpc"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"debug","ts":1627068195.7737236,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:47048","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cdn-Loop":["cloudflare"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"Cf-Ray":["6737483cfc550560-LAX"],"X-Forwarded-Proto":["https"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Te":["trailers"],"Cf-Visitor":["{\"scheme\":\"https\"}"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068196.8731592,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"error","ts":1627068206.8825853,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"debug","ts":1627068211.1728222,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:15256","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"Cf-Ray":["6737489d2c200560-LAX"],"Te":["trailers"],"X-Forwarded-Proto":["https"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Cdn-Loop":["cloudflare"],"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Content-Type":["application/grpc"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"Cf-Visitor":["{\"scheme\":\"https\"}"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068216.8680499,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}
{"level":"debug","ts":1627068222.7448807,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:2009","request":{"remote_addr":"108.162.215.40:33608","proto":"HTTP/2.0","method":"POST","host":"hw.itdiy.ml","uri":"/cdngrpc/Tun","headers":{"X-Forwarded-For":["218.6.202.25, 108.162.215.40"],"Te":["trailers"],"Content-Type":["application/grpc"],"User-Agent":["grpc-go/1.36.1"],"Cf-Connecting-Ip":["218.6.202.25"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["CN"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Cdn-Loop":["cloudflare"],"Cf-Ray":["673748e54be70560-LAX"],"X-Forwarded-Proto":["https"]}},"headers":{"Content-Type":["application/grpc"]},"status":200}
{"level":"error","ts":1627068226.8710802,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"context canceled"}

[imgk]

I’m not sure what caused this failure.

However, this error is weird.

{"level":"error","ts":1627067774.0379121,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}

I did some changes on dev branch. Can you test it?

[lxhao61]

I’m not sure what caused this failure.

However, this error is weird.

{"level":"error","ts":1627067774.0379121,"logger":"caddy.listeners.trojan","msg":"read prefix error: unexpected EOF"}

I did some changes on dev branch. Can you test it?

已测试dev分支的更改，问题解决了！