Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help on custom log #1626

Open
giopas opened this issue Dec 6, 2019 · 7 comments
Open

Help on custom log #1626

giopas opened this issue Dec 6, 2019 · 7 comments

Comments

@giopas
Copy link

@giopas giopas commented Dec 6, 2019

Hello,

I am trying to configure (in the config file) this type of log:

2019/12/06 08:16:01 my.domain.com 192.168.1.1 - - [06/Dec/2019:08:16:01 +0100] "GET /redirect.html?count=0.7408056120261262 HTTP/2.0" 200 548 "https://my.domain.com/" "Mozilla/5.0 (Linux; Android 9; ONEPLUS A5000) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Mobile Safari/537.36" 3
2019/12/06 08:24:42 my.domain.com 218.66.77.11 - - [06/Dec/2019:08:24:42 +0100] "GET /redirect.html?count=0.22918624993826597 HTTP/2.0" 200 548 "-" "Mozilla/5.0 (Linux; Android 9; ONEPLUS A5000) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Mobile Safari/537.36" 2
2019/12/06 17:36:54 my.domain.com 226.238.137.251 - - [06/Dec/2019:17:36:54 +0100] "GET /cgi-bin/login.html?1575650214 HTTP/1.1" 200 2653 "https://my.domain.com/redirect.html?count=0.282681040591797" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" 3

On Caddy the log is set to "{hostonly} {combined} {latency_ms}", but I do not know how to translate it to make it readable to goaccess.

How should I set it to work on goaccess.conf?

Thanks!

@giopas

This comment has been minimized.

Copy link
Author

@giopas giopas commented Dec 7, 2019

I somehow arrived to do so, but it still does not work:

log-format %d %t %v %h %^[%d/%b/%Y:%t %^] "%r" %s %b "%R" "%u" %^
date-format %Y/%b/%d
time-format %H:%M:%S
enable-panel REFERRERS
enable-panel KEYPHRASES

Any help would be really appreciated, as I am not good with log syntax.

Thanks!

@allinurl

This comment has been minimized.

Copy link
Owner

@allinurl allinurl commented Dec 8, 2019

Can you please describe what's the last field? Thanks

@giopas

This comment has been minimized.

Copy link
Author

@giopas giopas commented Dec 8, 2019

Thanks @allinurl , I think it is {latency_ms}, but I don't know if it is really necessary to keep it.

Reading here[1], it seems to correspond to %L. Correct?

Moreover, I see that in the log above there are two date standard used..

[1] https://lemanchet.fr/articles/optimizing-caddy-logs-for-goaccess.html

@giopas

This comment has been minimized.

Copy link
Author

@giopas giopas commented Dec 10, 2019

Hi @allinurl can you please have a look at the above? 1000 kudos!

EDIT: I managed to go one step further, but it still does not work:

log-format %Y/%m/%d %t %v %h %^[%d/%b/%Y:%t %^] "%r" %s %b "%R" "%u" %L
# date-format %Y/%m/%d
time-format %H:%M:%S
enable-panel REFERRERS
enable-panel KEYPHRASES

EDIT2: in case it may be useful, this is the entire config file:

### The port to which the connection is being attempted to connect.
port 7890

### Run GoAccess as daemon (if --real-time-html enabled).
daemonize true

### Log format type
log-format %Y/%m/%d %t %v %h %^[%d/%b/%Y:%t %^] "%r" %s %b "%R" "%u" %L
time-format %H:%M:%S
enable-panel REFERRERS
enable-panel KEYPHRASES

### Enable real-time HTML output.
real-time-html true

### Write  output to stdout given one of the following files and the
### corresponding extension for the output format:
output /share/Web/goaccess/index.html

# websocket connection
ws-url wss://wss.mydomain.com:443

### Specify the path to the input log file. If set, it will take
### priority over -f from the command line.
-log-file /share/Public/accesslogs/NAS1.log
-log-file /share/Public/accesslogs/NAS2.log

### GeoIP Options
geoip-database /share/Public/GeoIP/GeoLite2-City.mmdb

# Enable a list of user-agents by host. For faster parsing, do not
# enable this flag.
agent-list true
@giopas

This comment has been minimized.

Copy link
Author

@giopas giopas commented Dec 11, 2019

Hi @allinurl , I apologise for chasing you, but am I on a bad track? Could you please help me out? Thanks a bunch!

@allinurl

This comment has been minimized.

Copy link
Owner

@allinurl allinurl commented Dec 13, 2019

This should work:

goaccess log --log-format='%d %t %v %h %^[%^] "%r" %s %b "%R" "%u" %L' --date-format=%Y/%m/%d --time-format=%T
@giopas

This comment has been minimized.

Copy link
Author

@giopas giopas commented Dec 13, 2019

Indeed it works, thanks!

I only see the "keyphrases from Google's search" tab empty, but I think it is because the info is not available in the log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.