ManPage

LOGWARN(1)                  BSD General Commands Manual                  LOGWARN(1)

NAME
     logwarn -- utility for finding interesting messages in log files

SYNOPSIS
     logwarn [-achlnpqRvz] [-d dir | -f file] [-m firstpat] [-r sufpat]
             [-L maxlines] [-M maxprint] [-N maxerrors] logfile [-T num/secs]
             [!]pattern ...

     logwarn -i [-d dir | -f file] logfile

DESCRIPTION
     logwarn searches for interesting messages in log files, where ``interesting''
     is defined by an user-supplied list of positive and negative extended regular
     expressions provided on the command line.  A line that matches neither a posi-
     tive nor a negative pattern is considered a match (use the -p flag to change
     this default).

     Each log message is compared against each pattern in the order given.  Nega-
     tive patterns are specified with a ``!'' prefix.  If the log message matches a
     positive pattern before matching a negative !pattern, or if none of the pat-
     terns match, then it's printed to standard output.

     logwarn keeps track of its current position in the log file between invoca-
     tions, so each matching line is only ever output once.  This information is
     kept in a separate `state file' (see the -d and -f flags below).  By default,
     logwarn interprets a missing state file as if the log file were previously
     empty, resulting in a scan of its entire contents; this behavior can be
     changed with the -a and -i flags.

     logwarn will find messages in log files that have been rotated (and possibly
     compressed) since the previous invocation, as long as the rotated file has a
     name equal to logfile followed by a suffix matching the rotated log file suf-
     fix pattern (see the -r and -R flags below).

     By default, each line in the log file is considered to be a separate log mes-
     sage.  Log messages spanning multiple lines are supported with the use of the
     -m flag.

     If logfile is `-' then standard input is scanned.  In this mode, multiple
     invocations of logwarn can be pipelined together, optionally with intervening
     processing, to perform transformations of the raw log input prior to matching.
     Typically the -z flag is used in this scenario.

OPTIONS
     -a      Auto-initialize when the state file does not exist.  This flag turns
             on the -i intialization behavior in cases where logfile exists but the
             state file does not.  Normally this is not what you want, but this can
             be helpful in cases where it's important to avoid a flood of repeated
             log messages caused by state files somehow disappearing between invo-
             cations.

     -c      Match each pattern and firstpat (if any) case-insensitively.

     -d      Specify dir as the directory in which logwarn will store state infor-
             mation between invocations.  When this flag is used, the name of the
             state file is automatically generated from logfile.

             The default state directory is /var/lib/logwarn.

             It is an error to use this flag and -f at the same time.

     -f      Specify the state file used to store state information between invoca-
             tions.  Each logfile should have its own state file.

             To run in stateless mode, use -f /dev/null.

             It is an error to use this flag and -d at the same time.

     -h      Output help message and exit.

     -i      Initialize the saved state for logfile as `up to date'.  This causes
             the next invocation to start its scan at the current (as of this invo-
             cation) end of logfile.

     -L      Produce at most maxlines lines of output for any single log message;
             continuation lines beyond this limit are suppressed.  This flag is not
             needed unless multi-line messages are being detected using -m.

             Setting maxlines to zero has the same effect as -q.

     -l      Prefix each output line with the line number from the log file.

     -m      Enable multi-line support using firstpat to match the first line of
             new log messages.

             Without this flag, each line in logfile is considered a separate log
             message.  With this flag, each line in logfile is matched against the
             extended regular expression firstpat; non-matching lines are consid-
             ered continuations of the previous line.

             Multi-line mode will work correctly even if a message crosses a log
             file rotation boundary.

             Be careful with this flag: if you get the pattern wrong, it's possible
             nothing will ever match.

     -M      Output at most maxprint log messages; log messages beyond this limit
             are processed normally but their output is suppressed.

             Messages that span multiple lines (see -m) only count once.

             Setting maxprint zero has the same effect as -q.

     -N      Process at most maxerrors log messages.

             In contrast to -M, which limits the number of log messages output
             without affecting how many are processed, the -N flag causes logwarn
             to stop processing after encountering the specified number of log mes-
             sages, possibly before reaching the end of the log file.

             As a result, repeated invocations of logwarn may be required to output
             all outstanding log messages.  When using the -N flag, be sure to
             invoke logwarn frequently enough so that it doesn't fall behind.

             Messages that span multiple lines (see -m) only count once.

             The value of maxerrors must be at least one.

     -n      Normally, if the logfile does not exist, logwarn will exit with an
             error.  This flag causes logwarn to treat a non-existent logfile as if
             it were empty.

     -p      Change default match behavior to non-matching.  By default, if a log
             message doesn't match any of the positive or negative patterns, it is
             considered a match.  This flag reverses this behavior so that these
             messages are considered non-matches.

     -q      Disable the printing of matching log messages to standard output.  The
             process exit value can still be used to detect whether there were any
             matches (see below).

     -r      Make the extended regular expression sufpat the rotated log file suf-
             fix pattern.

             When logwarn detects that a log file has been rotated, it searches for
             the rotated log file by finding files in the same directory that have
             the same name as logfile plus a suffix matching the rotated log file
             suffix pattern.

             The default rotated log file suffix pattern is
             ^(-[[:digit:]]{8}|\.[01])(\.(gz|xz|bz2))?$

     -R      When multiple files match the rotated log file suffix pattern, norally
             the first one in sorting order is chosen.  When this flag is given,
             the last one is chosen instead.

             This option is appropriate when the suffix is formatted as a time-
             stamp.

     -T      Suppress output until num matching lines appear within a secs second
             interval.

             This flag comes after the logfile.  Each time it occurs, it applies to
             all subsequent patterns, or up until the next occurrence.  Use -T 1/0
             to revert to normal behavior, in which every matching occurrence is
             reported.

             The affected patterns are treated as a single group for counting pur-
             poses, i.e., a single counter tracks lines matching any of the pat-
             terns.  When the last of num matching lines is seen in a secs second
             window, it is printed, and the associated tracking is reset, so at
             least another num lines must be seen before another match is printed.

             Note that the calculated timing of occurrences is whenever logwarn
             reads the file.  Therefore logwarn should be invoked more frequently
             than your smallest secs time interval, to provide the required time
             resolution.

     -v      Output version information and exit.

     -z      Always start reading from the beginning of the file, even if the state
             file says otherwise.  This option is useful when reading from standard
             input.

DETAILS
     logwarn treats missing state files as if the log file were previously empty,
     and it never creates new state files for nonexistent log files.  The result is
     that when the -n flag is used, logwarn correctly functions even if the log
     file doesn't come into existence until after the first few runs.

     Log file rotation is detected by comparing filesystem inode numbers, so
     logwarn may exhibit incorrect behavior if (for example) an existing log file
     is replaced by a copy of itself.  In this situation, use the -i flag to reini-
     tialize the saved state.

     Currently, the supported compression types for rotated files are gzip(1),
     bzip2(1), and xz(1).  The corresponding executables gunzip(1), bunzip2(1), and
     unxz(1) must be present on the user's $PATH.

     When the -f flag is not specified, the state files in the state directory have
     names created by taking the logfile from the command line and replacing ``/''
     characters with ``_''.  Therefore, referring to the same log file using dif-
     ferent pathnames can result in inconsistent behavior.

     The matching patterns are only applied to the first line of a multi-line log
     message.  However, if the first line is a match, then entire log message
     including all continuation lines will be output.

     In order to avoid the race condition where logwarn reads a partially written
     line, if the last line of logfile does not end in a newline character, it is
     not processed.

     The maximum supported length for a single line is 100,000 characters; longer
     lines will be split and treated as multiple lines.

     If logwarn detects that a file's inode number has not changed but its size has
     decreased since the previous invocation, it assumes that the file has been
     truncated in place and scans from the beginning of the file.

EXAMPLES
     logwarn /var/log/warn

          Show all syslog warnings since the previous invocation.

     logwarn /var/log/warn | uniq -f 5 | logwarn -

          Same as above, compressing repeated instances of the same message into a
          single line of output.

     logwarn -p /var/log/apache2/access_log '^.* [^ ]+" 5[0-9]{2}'

          Show any Apache 5xx server errors since the previous invocation.

     logwarn -p -m '^myprog: ' '!retrying' 'ERROR'

          Show lines not containing `retrying' but containing `ERROR', as well as
          any subsequent lines in a multi-line log message, assuming the `myprog: '
          prefix marks the start of each new log message.

     logwarn /var/log/warn -T 3/60 pat1 -T 10/300 pat2 pat3

          Match three or more occurrences of pat1 within a one minute interval, or
          ten or more occurrences of either pat2 or pat3 within a five minute
          interval.

RETURN VALUES
     logwarn exits with one of the following values:

     0       No matching log messages were found.

     1       One or more matching log messages were found.

     2       An error occurred.

SEE ALSO
     Logwarn: Utility for finding interesting messages in log files,
     https://github.com/archiecobbs/logwarn.

AUTHOR
     Archie L. Cobbs <archie.cobbs@gmail.com>

BSD                               January 31, 2011                              BSD