mtail is not considering timestamp parsed from the log #271
Comments
sandeepkonda
changed the title
|
Can you check the mtail logs to see if it is getting a runtime parse error
on that regexp?
…On Fri, 27 Sep 2019, 20:54 Sandeep Konda, ***@***.***> wrote:
Hi,
I have used strptime to parse the timestamp from log, but mtail is still
using current system time as the time of the event instead of timestap from
log file.
I have built docker image using Dockerfile
<https://github.com/google/mtail/blob/master/Dockerfile> availble in
mtail repo.
Docker run command used,
docker run -it -p 3903:3903 -v
/Users/san/progs/tomcat.mtail:/progs/tomcat.mtail -v /Users/san/logs:/logs
mtail -logtostderr -progs /progs/tomcat.mtail -logs
/logs/localhost_access_log.txt --emit_metric_timestamp
Below is the output of mtail --version
mtail version v3.0.0-rc33-51-g5b5a874 git revision
5b5a874 go version go1.12.3 go arch amd64
go os linux
Content of mtail progam,
counter apache_http_requests_total by request_method, uri, request_status
counter apache_http_bytes_total by request_method, http_version,
request_status
/^/ +
/[(?P\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2} [+|-]\d{4})] / +
/(?P<client_hostname>[0-9A-Za-z.:-]+) / +
/(?P<server_hostname>[0-9A-Za-z.:-]+) / +
/(?P<remote_username>[0-9A-Za-z-\]+) / +
/- (?P\d{4}) / +
/(?P<request_method>[A-Z]+) (?P\S+) (?P<http_version>HTTP/[0-9.]+) / +
/(?P<request_status>\d{3}) / +
/(?P<response_size>\d+) / +
/(?P<response_time>\d+)/ +
/$/ {
strptime($timestamp, "02/Jan/2006:15:04:05 -0700")
apache_http_requests_total[$request_method][$uri][$request_status]++
$response_size > 0 {
apache_http_bytes_total[$request_method][$http_version][$request_status]
+= $response_size
}
}
Example Log line,
[17/Sep/2019:08:00:59 +0530] 10.222.45.26 10.222.45.26 CC\Administrator -
8444 GET /cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16
HTTP/1.1 200 105401 10
Above program is correctly parsing the log line, below is the link to
regex101 which verifies the regex,
https://regex101.com/r/2PmsWv/3
Output from /metrics endpoint
apache_http_requests_total{prog="tomcat.mtail",request_method="GET",request_status="200",uri="/cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16"}
1 1568687459000
Here timestamp 1568687459000 is the time when mtail ran instead of 17/Sep/2019:08:00:59
+0530 from log.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#271?email_source=notifications&email_token=AAXFX6ZIZ5VCT24VAFXJ3LTQLXQ7FA5CNFSM4I3E42HKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HOD44SA>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAXFX6ZV65HOPLCFWWGHLELQLXQ7FANCNFSM4I3E42HA>
.
|
|
Oh, in your program you haven't named the timestamp capture group, it looks
like?
…On Fri, 27 Sep 2019, 20:54 Sandeep Konda, ***@***.***> wrote:
Hi,
I have used strptime to parse the timestamp from log, but mtail is still
using current system time as the time of the event instead of timestap from
log file.
I have built docker image using Dockerfile
<https://github.com/google/mtail/blob/master/Dockerfile> availble in
mtail repo.
Docker run command used,
docker run -it -p 3903:3903 -v
/Users/san/progs/tomcat.mtail:/progs/tomcat.mtail -v /Users/san/logs:/logs
mtail -logtostderr -progs /progs/tomcat.mtail -logs
/logs/localhost_access_log.txt --emit_metric_timestamp
Below is the output of mtail --version
mtail version v3.0.0-rc33-51-g5b5a874 git revision
5b5a874 go version go1.12.3 go arch amd64
go os linux
Content of mtail progam,
counter apache_http_requests_total by request_method, uri, request_status
counter apache_http_bytes_total by request_method, http_version,
request_status
/^/ +
/[(?P\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2} [+|-]\d{4})] / +
/(?P<client_hostname>[0-9A-Za-z.:-]+) / +
/(?P<server_hostname>[0-9A-Za-z.:-]+) / +
/(?P<remote_username>[0-9A-Za-z-\]+) / +
/- (?P\d{4}) / +
/(?P<request_method>[A-Z]+) (?P\S+) (?P<http_version>HTTP/[0-9.]+) / +
/(?P<request_status>\d{3}) / +
/(?P<response_size>\d+) / +
/(?P<response_time>\d+)/ +
/$/ {
strptime($timestamp, "02/Jan/2006:15:04:05 -0700")
apache_http_requests_total[$request_method][$uri][$request_status]++
$response_size > 0 {
apache_http_bytes_total[$request_method][$http_version][$request_status]
+= $response_size
}
}
Example Log line,
[17/Sep/2019:08:00:59 +0530] 10.222.45.26 10.222.45.26 CC\Administrator -
8444 GET /cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16
HTTP/1.1 200 105401 10
Above program is correctly parsing the log line, below is the link to
regex101 which verifies the regex,
https://regex101.com/r/2PmsWv/3
Output from /metrics endpoint
apache_http_requests_total{prog="tomcat.mtail",request_method="GET",request_status="200",uri="/cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16"}
1 1568687459000
Here timestamp 1568687459000 is the time when mtail ran instead of 17/Sep/2019:08:00:59
+0530 from log.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#271?email_source=notifications&email_token=AAXFX6ZIZ5VCT24VAFXJ3LTQLXQ7FA5CNFSM4I3E42HKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HOD44SA>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAXFX6ZV65HOPLCFWWGHLELQLXQ7FANCNFSM4I3E42HA>
.
|
|
I have named it, I can see name timestamp in edit mode but somehow its getting removed by github on posting it. Same with port & uri, even they have been removed. Attached the program as docx document here. I don't see /tmp/mtail.INFO being created inside docker container or even on my machine. |
|
Let's find where this log has gone. What is the full command line you run
mtail with?
…On Sun, 29 Sep 2019, 13:05 Sandeep Konda, ***@***.***> wrote:
I have named it, I can see name timestamp in edit mode but somehow its
getting removed by github on posting it. Same with port & uri, even they
have been removed. Attached the program as docx document here.
tomcat program.docx
<https://github.com/google/mtail/files/3666288/tomcat.program.docx>
I don't see /tmp/mtail.INFO being created inside docker container or even
on my machine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#271?email_source=notifications&email_token=AAXFX66R5RQV35YH6BXKUDLQMALQVA5CNFSM4I3E42HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD73G4IY#issuecomment-536243747>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAXFX6ZWW5J3QJQLY4PDGOLQMALQVANCNFSM4I3E42HA>
.
|
|
I ran following commands to run mtail docker container on my Mac machine,
|
|
Thanks!
Please run mtail in your docker container with --help, and see where it
thinks the log directory is.
I think the flag is 'log_dir', and on Linux it should be /tmp.
It's possible that there's no /tmp to use in the container and so no log
gets created, and fails silently.
You could add another docker volume for /tmp to be sure.
…On Thu, 3 Oct 2019, 12:48 Sandeep Konda, ***@***.***> wrote:
I ran following commands to run mtail docker container on my Mac machine,
1. git clone https://github.com/google/mtail.git
2. cd mtail
3. docker build -t mtail .
4. docker run -it -p 3903:3903 -v
/Users/sankonda/Downloads/progs/tomcat.mtail:/progs/tomcat.mtail -v
/Users/sankonda/Downloads/logs:/logs mtail -logtostderr -progs
/progs/tomcat.mtail -logs /logs/localhost_access_log.txt
--emit_metric_timestamp
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#271?email_source=notifications&email_token=AAXFX64VAOFNG72MNE2PEQLQMVMRXA5CNFSM4I3E42HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAGZSYI#issuecomment-537762145>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAXFX66OL7X4S33VDZOJ7FLQMVMRXANCNFSM4I3E42HA>
.
|
|
I tried following things but still no success,
|
|
Ok the only other thing I can think of is to look at the status http page
on port 3903 by default. There should be a link to your program and then a
dump of its state plus the last runtime error.
…On Thu, 3 Oct 2019, 15:56 Sandeep Konda, ***@***.***> wrote:
I tried following things but still no success,
1.
Mapped /tmp volume, still mtail.INFO not created,
docker run -it -p 3903:3903 -v
/Users/sankonda/Downloads/progs/tomcat.mtail:/progs/tomcat.mtail -v
/Users/sankonda/Downloads/logs:/logs -v /Users/sankonda/Downloads/tmp:/tmp
mtail -logtostderr -progs /progs/tomcat.mtail -logs
/logs/localhost_access_log.txt --emit_metric_timestamp
2.
used -log_dir option, again mtail.INFO not created,
docker run -it -p 3903:3903 -v
/Users/sankonda/Downloads/progs/tomcat.mtail:/progs/tomcat.mtail -v
/Users/sankonda/Downloads/logs:/logs -v
/Users/sankonda/Downloads/info:/info mtail -logtostderr -progs
/progs/tomcat.mtail -logs /logs/localhost_access_log.txt
--emit_metric_timestamp -log_dir /info
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#271?email_source=notifications&email_token=AAXFX6ZZAB2UE256Z5BL7BLQMWCQDA5CNFSM4I3E42HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAHCWZA#issuecomment-537799524>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAXFX6YMO43RHLQO6Q5W44LQMWCQDANCNFSM4I3E42HA>
.
|
|
I verified http://localhost:3903/progz?prog=tomcat.mtail |
Did you solved this problem? |
No Gilbert, we had to move on to using telegraf due to few other considerations. |
|
there was insufficient information here to understand what the problem was |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I have used strptime to parse the timestamp from log, but mtail is still using current system time as the time of the event instead of timestap from log file.
I have built docker image using Dockerfile availble in mtail repo.
Docker run command used,
docker run -it -p 3903:3903 -v /Users/san/progs/tomcat.mtail:/progs/tomcat.mtail -v /Users/san/logs:/logs mtail -logtostderr -progs /progs/tomcat.mtail -logs /logs/localhost_access_log.txt --emit_metric_timestampBelow is the output of
mtail --versionmtail version v3.0.0-rc33-51-g5b5a874 git revision 5b5a8747c571f4573e68e3a0d38b747860c6f887 go version go1.12.3 go arch amd64 go os linuxContent of mtail progam,
counter apache_http_requests_total by request_method, uri, request_status
counter apache_http_bytes_total by request_method, http_version, request_status
/^/ +
/[(?P\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2} [+|-]\d{4})] / +
/(?P<client_hostname>[0-9A-Za-z.:-]+) / +
/(?P<server_hostname>[0-9A-Za-z.:-]+) / +
/(?P<remote_username>[0-9A-Za-z-\]+) / +
/- (?P\d{4}) / +
/(?P<request_method>[A-Z]+) (?P\S+) (?P<http_version>HTTP/[0-9.]+) / +
/(?P<request_status>\d{3}) / +
/(?P<response_size>\d+) / +
/(?P<response_time>\d+)/ +
/$/ {
strptime($timestamp, "02/Jan/2006:15:04:05 -0700")
apache_http_requests_total[$request_method][$uri][$request_status]++
$response_size > 0 {
apache_http_bytes_total[$request_method][$http_version][$request_status] += $response_size
}
}
Example Log line,
[17/Sep/2019:08:00:59 +0530] 10.222.45.26 10.222.45.26 CC\Administrator - 8444 GET /cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16 HTTP/1.1 200 105401 10Above program is correctly parsing the log line, below is the link to regex101 which verifies the regex,
https://regex101.com/r/2PmsWv/3
Output from /metrics endpoint
apache_http_requests_total{prog="tomcat.mtail",request_method="GET",request_status="200",uri="/cc/rest/en_US/reports/execute/3D0EFF5E1000016D007217F60A6B2D16"} 1 1568687459000Here timestamp
1568687459000is the time when mtail ran instead of17/Sep/2019:08:00:59 +0530from log.The text was updated successfully, but these errors were encountered: