Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
vearutop committed Sep 6, 2022
1 parent 10ca148 commit 8b24357
Showing 1 changed file with 31 additions and 4 deletions.
35 changes: 31 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# lograte

A small tool ([used to be](https://github.com/vearutop/lograte/blob/v0.1.0/main.go) ~25 lines of code) to calculate the
rate of lines in STDOUT and group by them by count with alphanumeric filter.
rate of lines in STDOUT and group them by count with alphanumeric filter.

## Installation

Expand Down Expand Up @@ -44,7 +44,11 @@ ssh -C log-collector.acme.com 'tail -f /var/log/app-error.log | grep "my feature
6862 lines since Mar 3 23:46:32, 605.47 per second
```

Or show top filtered messages by count or total size.
Or show top filtered messages by count or total size.

> **_NOTE:_** Filtered messages have all case insensitive sequences of `[a-z]-_%` with at least one digit or all digits replaced with `X`.
> This is usually enough to remove dynamic data from message and decrease cardinality.
> Filtered messages are collected in a limited number of buckets, once the limit is met all other messages are collected into the `Other` bucket.
```
tail -f /var/log/quick.log | lograte -top 5 -t 1s -by-size
Expand All @@ -54,8 +58,31 @@ tail -f /var/log/quick.log | lograte -top 5 -t 1s -by-size
foo-bar-18 i2 2022/09/03 08:22:17.206522 <recent log entry>
271382 lines since Sep 3 08:22:14, 90447.4 per second, 68.3 MB/s, 791 B/avg
------ Top 5 -------------------------
29810 lines, 9935.2 lps (11.0%), 14.7 MB/s (21.5%), 1546 B/avg: X X X/X/X X:X:X.X filtered entry
26559 lines, 8851.7 lps (9.8%), 9.1 MB/s (13.4%), 1083 B/avg: X X X/X/X X:X:X.X another filtered entry
29810 lines, 9935.2 lps (11.0%), 14.7 MB/s (21.5%), 1546 B/avg: X X X/X/X X:X:X.X filtered entry X.X
26559 lines, 8851.7 lps (9.8%), 9.1 MB/s (13.4%), 1083 B/avg: X X X/X/X X:X:X.X another filtered X entry
...
---------------------------------------
```

### Flags

```
lograte -help
```
```
Usage of lograte:
-buckets int
max number of buckets to track filtered messages (default 500)
-by-size
order messages by size instead of count
-len int
limit message length (default 120)
-line-buf int
line token buffer size (default 10000000)
-t duration
reporting interval (default 1s)
-top int
show top filtered messages ordered by rate
-version
print version and exit
```

0 comments on commit 8b24357

Please sign in to comment.