Swiss knife for logs
$ curl -sSL https://raw.githubusercontent.com/auhau/loggy/master/install.sh | bashdeb and rpm packages are build. Get the one for your OS
from latest release page and run sudo dpkg -i <package>.deb or
sudo rpm -i <package>.rpm.
If you've already got a Go development environment set up, you can grab it like this:
$ go get github.com/auhau/loggy$ brew tap auhau/loggy
$ brew install loggyscoop bucket add org https://github.com/auhau/scoop.git
scoop install org/loggy
Download on to your $PATH loggy binary for your operating system and architecture
from latest release page.
See loggy --help for this usage:
Loggy is a swiss knife for working with logs.
It stores the logs in internal buffer which allows it to do all sort of operations on top of them without the need to rerun the log's producing program or store them in file.
By default loggy reads from STDIN or you can specify file path to read the logs from specific file.
You quit the application by pressing Ctrl+C.
Configuration
-------------
All options that can be passed using CLI flags can be configured using config file or environment variables.
loggy looks for a config file ".loggy.toml" in current working directory and in $HOME folder.
Moreover you can use environment variables with "LOGGY_" prefix, where for example flag "--pattern" would be "LOGGY_PATTERN" env. variable.
The order of precedence is: $HOME config > CWD config > --config config > Env. variables > CLI flags.
Parsing pattern
---------------
The logs are parsed using parsing pattern that you have to configure in order to use filters. Internally regex is used for parsing. By default the input pattern is escaped for special characters, but if you need to do some advance parsing then you can disable the escaping using -r/--disable-regex-escape flag. You define parameters using syntax "<name:type>", where name is the name of parameter that you can refer to in filters and type is predefined type used to correctly find and parse the parameter. You can omit "type" in which case the "string" type will be used.
Lines that were not possible to parsed are colored with red color. Moreover counter of how many lines were not possible to parse is displayed in the status bar on the right end of it. It is only present if there are some lines that were not possible to parse.
There is built-in bool parameter called "patternMatches" reserved for marking log lines that were or were not possible to match against the parsing pattern. So you can use that to debug your parsing pattern with expressions like "!patternMatches"
Supported types:
- "string" defines string containing non-whitespace characters: [^\s]+
- "integer" defines a integer: [0-9]+
- "rest" collects the rest of the line: .*
Example log and bellow its parsing pattern:
[2022-09-11T15:04:22](authorization) DEBUG 200 We have received login information
[<timestamp:string>](<component:string>) <level:string> <code:integer> <message:rest>
Pattern names
-------------
In your config file you can create a [patterns] section where you can predefine your patterns using <name>="<pattern>" syntax and then use --pattern-name/-n flag to use it.
Filter
------
In order to use filter for the logs you have to define parsing pattern in which you define parameters that are extracted from the log lines. Then you can write filter expressions that will be applied on the logs. Filter has to return bool otherwise error will be shown.
loggy uses internally "expr" which has very rich set of arithmetic/string operators that you can use for your filters. Brief overview:
- modifiers: + - / * % **
- comparators: > >= < <= == !=
- logical ops: not ! or || and &&
- numeric constants, as 64-bit floating point (12345.678)
- numeric range: '..' (18..45)
- string constants (single or double quotes)
- string operators: + matches contains startsWith endsWith
- boolean constants: true false
- parenthesis to control order of evaluation ( )
- arrays e.g. [1, 2, 3]
- maps - e.g. {foo: "bar"}
- ternary conditional: ? :
- built in functions: len() all() none() any() one() filter() map() count()
For more details see: https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md
Example of filter for the parsing pattern log above:
level == "DEBUG" - display only debug messages
code in 400..500 - display logs with code in range from 400 to 500 (inclusive)
Keyboard shortcuts
------------------
Status bar on top displays several helpful information. Describing from left to right:
- Input name
- Optional "F" indicator that shows if loggy is following the end of the logs
- Filter status that displays "<number of filter matching lines>/<number of total lines>". If it has green background than filter is applied otherwise is turned off or not set.
- Optional number of lines that were not possible to match against the parsing pattern.s
Main key shortcuts:
- "/" for setting filter
- "f" for toggling filter
- "p" for setting parsing pattern input
- "h" for displaying help
Logs navigation:
- "j", "k" or arrow keys for scrolling by one line
- "g" to move to top
- "G" to move to bottom and follow bottom
- "Ctrl-F", "page down" to move down by one page
- "Ctrl-B", "page up" to move up by one page
- Mouse scrolling also scrolls the view accordingly
Input fields:
- Left arrow: Move left by one character.
- Right arrow: Move right by one character.
- Home, Ctrl-A, Alt-a: Move to the beginning of the line.
- End, Ctrl-E, Alt-e: Move to the end of the line.
- Alt-left, Alt-b: Move left by one word.
- Alt-right, Alt-f: Move right by one word.
- Backspace: Delete the character before the cursor.
- Delete: Delete the character after the cursor.
- Ctrl-K: Delete from the cursor to the end of the line.
- Ctrl-W: Delete the last word before the cursor.
- Ctrl-U: Delete the entire line.
Usage:
loggy [path to log file] [flags]
Flags:
-b, --buffer-size int number of lines that will be buffered (default 10000)
--config string config file (default is $HOME/.loggy.yaml)
-r, --disable-regex-escape turn off pattern regex escaping
-f, --follow turn on following mode which always show latest logs
-h, --help help for loggy
-p, --pattern string parsing pattern see above for details
-n, --pattern-name string use predefined pattern in config
PRs are welcome! If you plan to make some bigger change then please discuss it previously in Issue so there would not be any misunderstanding and your work would get merged!