CLI tool for parsing NGINX access log and filtering by fields or values with output to JSON or plain text
Pre-alpha, don't use on production.
git clone https://github.com/creadone/ngx && cd ngx && shards build --release
The main difference between the filter
and find
that the filter
returns only subset of the fields and nothing more, and the find
can compare by value of fields but always returns all fields.
Fields
ip, user, day, month, year, hour, minute, second, timezone, method, path, protocol, version, code, bytes, referer, user_agent
Input logs
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
Run ngx
cat access.log | ngx -m find -f ip=127.0.0.1, method=GET -o json
Output
{"ip":"127.0.0.1","user":"-","day":"30","month":"Oct","year":"2020","hour":"10","minute":"27","second":"00","timezone":"+0000","method":"GET","path":"/","protocol":"HTTP","version":"1.1","code":"200","bytes":"2205","referer":"-","user_agent":"ApacheBench/2.3"}
{"ip":"127.0.0.1","user":"-","day":"30","month":"Oct","year":"2020","hour":"10","minute":"27","second":"00","timezone":"+0000","method":"GET","path":"/","protocol":"HTTP","version":"1.1","code":"200","bytes":"2205","referer":"-","user_agent":"ApacheBench/2.3"}
{"ip":"127.0.0.1","user":"-","day":"30","month":"Oct","year":"2020","hour":"10","minute":"27","second":"00","timezone":"+0000","method":"GET","path":"/","protocol":"HTTP","version":"1.1","code":"200","bytes":"2205","referer":"-","user_agent":"ApacheBench/2.3"}
Input logs
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
127.0.0.1 - - [30/Oct/2020:10:27:00 +0000] "GET / HTTP/1.1" 200 0 "-" "ApacheBench/2.3"
Run ngx
car access.log | ngx -m filter -f ip,method,path -o json
Output
{"ip":"127.0.0.1","method":"GET","path":"/"}
{"ip":"127.0.0.1","method":"GET","path":"/"}
{"ip":"127.0.0.1","method":"GET","path":"/"}
{"ip":"127.0.0.1","method":"GET","path":"/"}
- Tests
- Merge
finder
andfilter
into one - Custom parsing templates
- Add expressions to comparison:
cat access.log | ngx -m find -f code > 400, ip << 127.0.0.1/24 -o json
- Export output with TCP or HTTP
- Fork it (https://github.com/creadone/ngx/fork)
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request
- Sergey Fedorov - creator and maintainer