mtail is tailing only the first file matching the glob pattern

[sandeepdharembra]

./mtail -version
mtail version v3.0.0-rc29 git revision b5aa95d go version go1.11.6 go arch amd64 go os linux

Log Files I am Trying to get metrics from -
I basically need to tail the latest file which in below list is the from 25th

1717865193 Apr 24 00:00 access-20190423.log
1734198453 Apr 25 00:00 access-20190424.log
377365821 Apr 25 05:11 access-20190425.log

Pattern used for log files -

access-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].log

mtail command used -

mtail_v3.0.0-rc29_linux_amd64 --progs programs --logs access-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].log

INFO Log output -

Log file created at: 2019/04/25 05:05:05
Running on machine: xxxxxxx
Binary: Built with gc go1.11.6 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0425 05:05:05.128509   19224 main.go:94] mtail version v3.0.0-rc28 git revision a3c298744d8073bd1bcdd7d9e4c7ef2c6309f2b4 go version go1.11.6 go arch amd64 go os linux
I0425 05:05:05.128713   19224 main.go:95] Commandline: ["/xxxxx/mtail_v3.0.0-rc29_linux_amd64" "--progs" "/xxxxxx/programs" "--logs" "/xxxx/access-20190423.log" "/xxxx/access-20190424.log" "xxxx/access-20190425.log" "-v=2"]
.
.
.
.
I0425 05:05:05.131701   19224 loader.go:219] Loaded program accesslogs.mtail
I0425 05:05:05.131707   19224 loader.go:238] Program accesslogs.mtail has goroutine marker 0x61636365
I0425 05:05:05.131733   19224 vm.go:749] Starting program accesslogs.mtail
I0425 05:05:05.131738   19224 loader.go:242] Started accesslogs.mtail
**I0425 05:05:05.131825   19224 tail.go:246] Tailing /xxxx/access-20190423.log**
I0425 05:05:05.131956   19224 store.go:136] Starting metric store expiry loop every 1h0m0s
I0425 05:05:05.131967   19224 tail.go:397] Starting log handle expiry loop every 1h0m0s
I0425 05:05:05.131980   19224 mtail.go:313] Listening on [::]:3903

Browser output -

mtail on :3903

Build: mtail version v3.0.0-rc29 git revision b5aa95d go version go1.11.6 go arch amd64 go os linux

Metrics: json, prometheus, varz

Debug: debug/pprof, debug/vars

program name	errors	load errors	load successes	runtime errors
accesslogs.mtail	No compile errors		1

Log Tailer
Patterns

/xxxxxxx/access-20190423.log

Log files watched

pathname	errors	rotations	truncations	lines read
/xxxxxxxxxxxxxxxxxx/access-20190423.log

EXPECTED -

It should tail all the files in the folder (the ones from 23,24 and 25) and since the one from 25 is the latest one and has logs written to it, push data from this to :3903/metrics endpoint

ACTUAL

It shows tailing the file from 23 which is the first one in the list of files matching the glob pattern as can be seen from INFO logs. Since this file is not being written, I don't see any metrics at :3903/metrics

[jaqx0r]

The problem is that the pattern you provided got interpreted by the shell, and then expanded to three files; then mtail only saw the first one because it ignores arguments without flags.

The fix above now aborts mtail if you give extra arguments, so you'd be able to detect when this happens.

You need to quote the pattern so the shell doesn't interpret it, and then mtail will be able to see new files that match the pattern that are created after mtail starts.

[sandeepdharembra]

Thanks @jaqx0r,

I was actually able to workaround this yesterday when I included the command in a shell script. mtail was able to pick all the files. Will keep in mind to use quotes

Thanks again for your time