Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

caddy 的内存占用异常高 #610

Closed
zwyyy456 opened this issue Jan 9, 2024 · 2 comments
Closed

caddy 的内存占用异常高 #610

zwyyy456 opened this issue Jan 9, 2024 · 2 comments

Comments

@zwyyy456
Copy link

zwyyy456 commented Jan 9, 2024

caddy 一共使用了两个插件,一个是 caddy-trojan ,另一个是 naive-proxy 作者修改的 forwardproxy 插件,几个 vps ,凡是按照上面的配置文件部署了 caddy 的,内存占用都是起码 500-600M ,我 24g 的甲骨文 arm 机器,caddy 内存占用甚至达到了 5-6g ,问了下 chatgpt ,也没给出什么实质性的建议;

重启 caddy 之后,内存占用倒是降下来了,但是似乎随着时间增长,占用会逐渐升高,请问要如何排查问题呢?Caddyfile 的内容如下,对用户名、密码、域名等敏感信息作了修改;

{
    order trojan before file_server
    servers :4443 {
        listener_wrappers {
            trojan
        }
    }
    log {
        output file /var/log/caddy/default.log {
            roll_size 10MiB
        }
    }
    trojan {
        caddy
        no_proxy
        users user tro_password
    }
}
tro.arm-br.example.com {
    reverse_proxy https://tro-arm-br.example.com:4443
}
naive.arm-br.example.com {
    reverse_proxy https://arm-br.example.com:443 
}
plex.arm-br.example.com {
    reverse_proxy arm-br.example.com:32400
}
qbit.arm-br.example.com {
    reverse_proxy arm-br.example.com:28080
}
cd2.arm-br.example.com {
    reverse_proxy arm-br.example.com:19798
}
status.arm-br.example.com {
    reverse_proxy arm-br.example.com:10182
}

:443, arm-br.example.com {
    tls user456@gmail.com
    route {
        forward_proxy {
            basic_auth user naive_password
            hide_ip
            hide_via
            probe_resistance
        }
        file_server {
            root /usr/share/caddy
        }
    }
}
:4443, tro-arm-br.example.com {
tls user456@gmail.com 
    route {
        trojan {
            connect_method
            websocket
        }
        file_server {
            root /usr/share/caddy
        }
    }
}

:6443, arm-brv6.example.com {
    tls user456@gmail.com
    route {
        forward_proxy {
            basic_auth user naive_password
            hide_ip
            hide_via
            probe_resistance
        }
        file_server {
            root /usr/share/caddy
        }
    }
}

这里贴出一部分日志

{"level":"info","ts":1704763488.7812674,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1704763488.7820559,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7821221,"logger":"http","msg":"enabling HTTP/3 listener","addr":":4443"}
{"level":"info","ts":1704763488.7824142,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7824407,"logger":"http","msg":"enabling HTTP/3 listener","addr":":6443"}
{"level":"info","ts":1704763488.7826765,"logger":"http.log","msg":"server running","name":"srv2","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7827032,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704763488.7827075,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["naive.lw.ggll.eu.org","plex.lw.ggll.eu.org","qbit.lw.ggll.eu.org","tro.lw.ggll.eu.org","cd2.lw.ggll.eu.org","tro-lw.ggll.eu.org","lwv6.ggll.eu.org","status.lw.ggll.eu.org"]}
{"level":"warn","ts":1704763488.8449705,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","instance":"f6b0280a-020c-47f0-afa4-168ce77cd684","try_again":1704849888.8449664,"try_again_in":86399.999999489}
{"level":"info","ts":1704763488.8450851,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1704763488.8468266,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":1704763488.8481233,"msg":"serving initial configuration"}
{"level":"error","ts":1704763522.5706208,"logger":"caddy.listeners.trojan","msg":"read prefix error: read tcp 112.15.179.204:55125 -> 38.72.148.44:4443: read: EOF"}
{"level":"error","ts":1704768338.0250442,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704771357.2400272,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772503.677092,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: read tcp 38.72.148.44:4443->167.248.133.127:43178: read: connection reset by peer"}
{"level":"error","ts":1704772503.7557719,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704772506.9289355,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772510.378716,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704772510.4845634,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: tls: first record does not look like a TLS handshake"}
{"level":"error","ts":1704775922.9165707,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44'"}
{"level":"error","ts":1704779861.6374896,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for 'naive-lw.ggll.eu.org'"}
{"level":"error","ts":1704779861.9872792,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for 'naive-lw.ggll.eu.org'"}
{"level":"error","ts":1704784934.799494,"logger":"http.log.error","msg":"dial tcp 38.72.148.44:10182: connect: connection refused","request":{"remote_ip":"104.237.134.123","remote_port":"43238","client_ip":"104.237.134.123","proto":"HTTP/1.1","method":"GET","host":"status.lw.ggll.eu.org","uri":"/","headers":{"Accept-Encoding":["gzip"],"Connection":["close"],"User-Agent":["Go-http-client/1.1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"status.lw.ggll.eu.org"}},"duration":0.00567612,"status":502,"err_id":"evhj4586b","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
{"level":"error","ts":1704785624.133557,"logger":"http.log.error","msg":"dial tcp 38.72.148.44:28080: connect: connection refused","request":{"remote_ip":"139.177.207.147","remote_port":"58692","client_ip":"139.177.207.147","proto":"HTTP/1.1","method":"GET","host":"qbit.lw.ggll.eu.org","uri":"/","headers":{"Accept-Encoding":["gzip"],"Connection":["close"],"User-Agent":["Go-http-client/1.1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","server_name":"qbit.lw.ggll.eu.org"}},"duration":0.276857775,"status":502,"err_id":"zr76ydb3z","err_trace":"reverseproxy.statusError (reverseproxy.go:1267)"}
{"level":"info","ts":1704789498.6327744,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1704789498.6343508,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1704789498.6344306,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1704789498.6832166,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1704789498.6832995,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1704789499.0957417,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1704789499.0968235,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1704789499.096848,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1704789499.0968544,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
{"level":"info","ts":1704789499.0968592,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv2"}
{"level":"info","ts":1704789499.0977018,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.097743,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1704789499.0981522,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.098187,"logger":"http","msg":"enabling HTTP/3 listener","addr":":4443"}
{"level":"info","ts":1704789499.0985396,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.0985725,"logger":"http","msg":"enabling HTTP/3 listener","addr":":6443"}
{"level":"info","ts":1704789499.0986316,"logger":"http.log","msg":"server running","name":"srv2","protocols":["h1","h2","h3"]}
{"level":"info","ts":1704789499.0986369,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["tro-lw.ggll.eu.org","lwv6.ggll.eu.org","status.lw.ggll.eu.org","naive.lw.ggll.eu.org","plex.lw.ggll.eu.org","qbit.lw.ggll.eu.org","tro.lw.ggll.eu.org","cd2.lw.ggll.eu.org"]}
{"level":"info","ts":1704789499.1366014,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004d9b80"}
{"level":"info","ts":1704789499.1503842,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":1704789499.1612566,"msg":"serving initial configuration"}
{"level":"info","ts":1704789499.1748154,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
{"level":"info","ts":1704789499.176846,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"error","ts":1704790385.6787434,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"lw.ggll.eu.org:19798","duration":0.000933625,"request":{"remote_ip":"112.15.179.204","remote_port":"4395","client_ip":"112.15.179.204","proto":"HTTP/3.0","method":"GET","host":"cd2.lw.ggll.eu.org","uri":"/_framework/icudt.dat","headers":{"X-Forwarded-For":["112.15.179.204"],"X-Forwarded-Proto":["https"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Dest":["empty"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.54"],"Cache-Control":["max-age=0"],"Dnt":["1"],"Accept-Language":["zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7"],"Sec-Fetch-Site":["same-origin"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://cd2.lw.ggll.eu.org/service-worker.js"],"X-Forwarded-Host":["cd2.lw.ggll.eu.org"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"cd2.lw.ggll.eu.org"}},"error":"writing: H3_REQUEST_CANCELLED"}
{"level":"error","ts":1704790385.6794074,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"lw.ggll.eu.org:19798","duration":0.000900173,"request":{"remote_ip":"112.15.179.204","remote_port":"4395","client_ip":"112.15.179.204","proto":"HTTP/3.0","method":"GET","host":"cd2.lw.ggll.eu.org","uri":"/_framework/icudt_no_CJK.dat","headers":{"Dnt":["1"],"Accept-Language":["zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.54"],"Accept":["*/*"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["cd2.lw.ggll.eu.org"],"Sec-Fetch-Dest":["empty"],"X-Forwarded-For":["112.15.179.204"],"Referer":["https://cd2.lw.ggll.eu.org/service-worker.js"],"Sec-Fetch-Mode":["cors"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Site":["same-origin"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"cd2.lw.ggll.eu.org"}},"error":"writing: H3_REQUEST_CANCELLED"}
{"level":"error","ts":1704790488.9171484,"logger":"caddy.listeners.trojan","msg":"read prefix error, not io, rewind and let normal caddy deal with it: no certificate available for '38.72.148.44:4443'"}

本人不太懂 go 与网络的相关知识,还请作者有空的时候能帮忙看看,或者给出一点排查问题的建议。

@zwyyy456
Copy link
Author

zwyyy456 commented Jan 9, 2024

假设被转发给的端口,例如 10182,在该服务器上并没有应用监听,会有什么负面效果吗?

@klzgrad
Copy link
Owner

klzgrad commented Jan 10, 2024

内存泄漏,并不容易定位问题,建议你设置定时重启

@klzgrad klzgrad closed this as not planned Won't fix, can't repro, duplicate, stale Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants