Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse custom nginx JSON log #2438

Closed
mannickutd opened this issue Jan 7, 2023 · 2 comments
Closed

Parse custom nginx JSON log #2438

mannickutd opened this issue Jan 7, 2023 · 2 comments

Comments

@mannickutd
Copy link

mannickutd commented Jan 7, 2023

I have a simple log but I cannot figure out how to parse the datetime component

sample line

{"time":1673085237.244,"client":"10.10.10.10","method":"GET","request":"GET /search/? HTTP/1.1","request_length":390,"status":"307","bytes_sent":238,"gzip_ratio":"","body_bytes_sent":0,"referer":"","user_agent":"python-requests/2.28.1","accept_encoding":"gzip","accept":"*/*","request_time":0.002}

goaccess sample_access_logs.txt -o report.html --date-format %* --time-format %* --log-format '{"time":%T,"client":"%h","method":"%m","request":"%r","request_length":%^,"status":"%s","bytes_sent":"%b","gzip_ratio":"%^","body_bytes_sent":%^,"referer":"%R","user_agent":"%u","accept_encoding":"%^","accept":"%^","request_time":%L}'

This produces the output

A valid date is required
Format Errors - Verify your log/date/time format

If I change it to

goaccess sample_access_logs.txt -o report.html --date-format %* --time-format %* --log-format '{"time":%d,"client":"%h","method":"%m","request":"%r","request_length":%^,"status":"%s","bytes_sent":"%b","gzip_ratio":"%^","body_bytes_sent":%^,"referer":"%R","user_agent":"%u","accept_encoding":"%^","accept":"%^","request_time":%L}'


Token '1673085237.244' doesn't match specifier '%d'
@allinurl allinurl changed the title Parse nginx json log Parse custom nginx JSON log Jan 8, 2023
@allinurl
Copy link
Owner

allinurl commented Jan 8, 2023

This should do it:

goaccess access.log --log-format='{ "time": "%x.%^", "client": "%h", "method": "%m", "request": "%U", "status": "%s", "bytes_sent": "%b", "referer": "%R", "user_agent": "%u", "request_time": "%T" }' --datetime-format='%s'

@allinurl
Copy link
Owner

Closing this. Feel free to reopen it if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants