can't pipe awk to goaccess

[hmmatos]

Hi,

i'm trying to pipe awk command to goaccess but is not working

awk '$7~/\<api\>/' /var/log/nginx/access.log | goaccess

is anybody experience the same problem? or i'm doing something wrong?

thanks

[sabin800]

I'm having the same issue, however I'm trying to "gunzip -c" a zipped apache log file and keep getting a goaccess specific error. Does goaccess not accept STDIN?

[vmanyushin]

try to comment log-file option in config file.

[hmmatos]

Hi @vmanyushin,

if I comment the log-file in config file he pipe the awk but Break Out and the debug-file only says "bye"

[allinurl]

Can you please post the error message you are getting and if possible a few lines from your access log matching the pattern posted above?

[hmmatos]

I don't get any error, please see the video below
https://drive.google.com/open?id=0B5VlM_LKLNzGQk52Z0xhczZaaWM

nginx log

192.168.94.58 - - [25/Aug/2016:14:39:19 +0100] "PUT /api/list/3/contact/1111111111/field/status HTTP/1.1" 200 56 "http://site.com/ui/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0"0.882

[allinurl]

@hmmatos what output do you get on the specified report file? i.e., /var/www/services.e... Also have you tried running goaccess without -o and outputting to the terminal?

[allinurl]

@sabin800 can you please post the command you are running and the output from goaccess so I can take a look? Thanks

[hmmatos]

@allinurl what do you mean by the output report file? is a html file generated by goaccess.
if I run goaccess without -o and outputting to the terminal it work but is a static report I have to quit and execute again to update.

[allinurl]

@hmmatos if I run it with the sample line you posted:

awk '$7~/\<api\>/' sample.log | goaccess --log-format=COMBINED -o report.html

I get the correct html report. I'm not sure what issue is that you are having?

[hmmatos]

@allinurl I'm getting the same result, please see the video below
https://drive.google.com/open?id=0B5VlM_LKLNzGdlZUT2d4VTNINTQ

is the log too large?

[allinurl]

Is the log too large? not at all.

The command you are running should ouput an html file called report.html. However, what I see is that you are outputting a real-time report, while piping data in. Right now piping a log to GoAccess will disable the real-time functionality. This is part of issue #428.

[hmmatos]

@allinurl ok that's the problem.
I will keep my eyes on that issue

Thanks

[allinurl]

@hmmatos in the meantime, a workaround is to filter your dataset to a new log file and use that to feed goaccess.

I'll close this issue since it's related to #428 and #117.

Stay tuned for these changes!

[allinurl]

@hmmatos I've pushed a commit that enables the ability to output live stats from a stream or an unclosed STDIN, i.e., tail -f as well as parsing multiple logs in live mode:

goaccess --log-format=COMBINED access.log access.log.1

or

tail -f access.log | goaccess --log-format=COMBINED -

or even

tail -f access.log | goaccess --log-format=COMBINED -o report.html --real-time-html -

It also opens the possibility for live data filtering from the pipe such as:

tail -f access.log | grep -i --line-buffered 'firefox' | goaccess --log-format=COMBINED -

Feel free to build from master to test this out, otherwise it will be pushed out in the upcoming release. Thanks.

[hmmatos]

@allinurl thanks for the commit i'll tested