Caddy log format - client_ip should be used instead of remote_ip

[fliespl]

Currently goaccess uses remote_ip in Caddy json format, but according to docs client_ip should be used since it's the one being set with proper ips when using proxies.

https://caddyserver.com/docs/caddyfile/matchers#client-ip

[allinurl]

Hey, thanks for letting me know! Do you mind sharing a few lines from your access log so I can check out the latest changes?

[fliespl]

Direct access without loadbalancer / proxy:

{"level":"info","ts":1703415994.3220897,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_ip":"X.X.X.X","remote_port":"49570","client_ip":"X.X.X.X","proto":"HTTP/1.1","method":"GET","host":"Y.Y,Y,Y:8080","uri":"/index.php","headers":{"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7,fr;q=0.6,es;q=0.5,it;q=0.4,ar;q=0.3,zh-CN;q=0.2,zh;q=0.1,da;q=0.1,sv;q=0.1,nl;q=0.1,fi;q=0.1,de;q=0.1,el;q=0.1,id;q=0.1,he;q=0.1,ja;q=0.1,ms;q=0.1,ru;q=0.1,pt;q=0.1"],"Connection":["keep-alive"],"Dnt":["1"]}},"bytes_read":0,"user_id":"","duration":0.000010499,"size":0,"status":0,"resp_headers":{"Server":["Caddy"]}}

client_ip = remote_ip = my public IP

Using loadbalancer / proxy:

{"level":"info","ts":1703416013.6445792,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"172.71.190.23","remote_port":"48104","client_ip":"X.X.X.X","proto":"HTTP/2.0","method":"GET","host":"xxx","uri":"/abc","headers":{"X-Forwarded-Proto":["https"],"Sec-Ch-Ua":["\"Not_A Brand\";v=\"8\", \"Chromium\";v=\"120\", \"Google Chrome\";v=\"120\""],"Dnt":["1"],"Referer":["https://xxx/"],"Cookie":[],"Cf-Connecting-Ip":["X.X.X.X"],"Cdn-Loop":["cloudflare"],"If-Modified-Since":["Wed, 18 Jan 2023 11:16:33 GMT"],"If-None-Match":["\"roohzl6az\""],"Sec-Ch-Ua-Platform":["\"Windows\""],"Accept":["*/*"],"Accept-Language":["pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7,fr;q=0.6,es;q=0.5,it;q=0.4,ar;q=0.3,zh-CN;q=0.2,zh;q=0.1,da;q=0.1,sv;q=0.1,nl;q=0.1,fi;q=0.1,de;q=0.1,el;q=0.1,id;q=0.1,he;q=0.1,ja;q=0.1,ms;q=0.1,ru;q=0.1,pt;q=0.1"],"Cf-Ipcountry":["PL"],"X-Forwarded-For":["X.X.X.X"],"Cf-Ray":["83a860a2bba70008-IAD"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"],"Sec-Fetch-Site":["same-origin"],"Accept-Encoding":["gzip"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Dest":["script"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"xxx"}},"bytes_read":0,"user_id":"","duration":0.000209214,"size":0,"status":304,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000"],"Access-Control-Allow-Origin":["*"],"Etag":["\"roohzl6az\""]}}

remote_ip=172.71.190.23 is cloudflare IP here, while client_ip=X.X.X.X is my own public IP.

[Teqed]

I've edited my goaccess.conf file, usually found in /etc/goaccess/goaccess.conf

time-format %s
date-format %s
...
# CADDY JSON Structured
-#log-format {"ts":"%x.%^","request":{"remote_ip":"%h","proto":"%H","method":"%m","host":"%v","uri":"%U","headers":{"User-Agent":["%u","%^"]},"tls":{"cipher_suite":"%k","proto": "%K"}},"duration": "%T","size": "%b","status": "%s","resp_headers":{"Content-Type":["%M;%^"]}}
+log-format {"ts":"%x.%^","request":{"client_ip":"%h","proto":"%H","method":"%m","host":"%v","uri":"%U","headers":{"User-Agent":["%u"],"Referer":["%R"]},"tls":{"cipher_suite":"%k","proto": "%K"}},"duration": "%T","size": "%b","status": "%s","resp_headers":{"Content-Type":["%M"]}}

Take note that more had to be changed than just remote_ip to client_ip, it seems like the Caddy JSON preset found here handles the headers differently than the comment in the config file.

Then I can happily view my logs with visitor IPs logged from my trusted proxy's headers.

goaccess /var/log/caddy/access.log -o /var/www/html/goaccess/report.html --real-time-html

The default config file's comment for Caddy JSON should probably be updated to handle headers the same way as the preset -- and then it would be trivial for someone to uncomment it and swap client_ip if they were using a trusted proxy.

[allinurl]

Fixed on #2629