@darold darold released this Sep 13, 2018 · 18 commits to master since this release

Assets 2

2018-09-12 - v10.1

This release of pgBadger is a maintenance release that fixes reports
in incremental mode and multiprocess with -j option. Log parsing from
standard input was also broken. If you are using v10.0 please upgrade
now.

  • Add test on pgbouncer log parser.
  • Some little performances improvment.
  • Fix not a valid file descriptor at pgbadger line 12314.
  • Fix unwanted newline in progressbar at startup.
  • Remove circleci files from the project.
  • Remove dependency of bats and jq for the test suite, they are
    replaced with Test::Simple and JSON::XS.
  • Add more tests especially for incremental mode and input from
    stdin that was broken in release 10.0.
  • Sync pgbadger, pod, and README, and fix some syntax errors.
    Thanks to Christoph Berg for the patch.
  • Add documentation on how to install Perl module JSON::XS from
    apt and yum repositories.
  • Fix URI for CSS in incremental mode. Thanks to Floris van Nee
    for the report.
  • Fix fatal error when looking for log from STDIN. Thanks to
    Jacek Szpot for the report.
  • Fixes SED use for OSX builds. Thanks to Steve Newson for the
    patch.
  • Fix illegal division by zero in incrental mode. Thanks to
    aleszeleny for the report.
  • Replace SQL::Beautify with v3.1 of pgFormatter::Beautify.

Thanks a lot to Etienne Bersac from Dalibo who have initiate the
pgbadger test suite and added information about contributing in
the documentation. Unitary tests are still not complete but tests
of features fixed in this maintenance release have been added.

@darold darold released this Sep 9, 2018 · 36 commits to master since this release

Assets 2

2018-09-09 - v10.0

This release of pgBadger is a major release that adds some new
features and fix all issues reported by users since last release.

  • Add support of pgbouncer syslog log file format.
  • Add support to all auto_explain format (text, xml, json and yaml).
  • Add support to %q placeholder in log_line_prefix.
  • Add jsonlog format of Michael Paquier extension, with -f jsonlog pgbadger will be able to parse the log.
  • Replace the SQL formatter/beautify with v3.0 of pgFormatter.

There is some new command line option:

  • Add --prettify-json command line option to prettify JSON output.
  • Add --log-timezone +/-XX command line option to set the number of hours from GMT of the timezone that must be used to adjust date/time read from log file before beeing parsed. Note that you might still need to adjust the graph timezone using -Z when the client has not the same timezone.
  • Add --include-time option to add the ability to choose times that you want to see, instead of excluding all the times you do not want to see (--exclude-time).

The pgBadger project and copyrights has been transfered from Dalibo
to the author and official maintainer of the project. Please update
your links:

I want to thanks the great guys at Dalibo for all their investments
into pgBadger during these years and especially Damien Clochard and
Jean-paul argudo for their help to promote pgBadger.

Here is the complete list of changes and acknowledgement in this
release.

  - Fix checkpoint distance and estimate not reported in incremental
    mode. Thanks to aleszeleny for the report.
  - Fix title of pgbouncer simultaneous session report. Thanks to
    Jehan Guillaume De Rorthais for the report.
  - Add support of pgbouncer syslog log file format. Thanks to djester
    for the feature request.
  - Fix error when a remote log is empty. Thanks to Parasit Hendersson
    for the report.
  - Fix test with binary format. Binary file must be generated as it
    is dependent of the plateform. Thanks to Michal Nowak for the
    report.
  - Fix case where an empty explain plan is generated.
  - Fix parsing of autodetected default format with prefix in command
    line.
  - Remove dependency of git command in Makefile.PL.
  - Update documentation about options changes and remove of the
    [%l-1] part of the mandatory prefix.
  - Fix parsing of vacuum / analyze system usage for PostgreSQL 10.
    Thanks to Achilleas Mantzios for the patch.
  - Fix Temporary File Activity table.
  - Remove dependency to git during install.
  - Add --log-timezone  +/-XX  command line option to set the number
    of hours from GMT of the timezone that must be used to adjust
    date/time read from log file before beeing parsed. Using this
    option make more difficult log search with a date/time because the
    time will not be the same in the log. Note that you might still
    need to adjust the graph timezone using -Z when the client has not
    the same timezone. Thanks to xdexter for the feature request and
    Julien Tachoire for the patch.
  - Add support to auto_explain json output format. Thanks to dmius
    for the report.
  - Fix auto_explain parser and queries that was counted twice.
    Thanks to zam6ak for the report.
  - Fix checkpoint regex to match PostgreSQL 10 log messages. Thanks
    to Edmund Horner for the patch.
  - Update description of -f | --format option by adding information
    about jsonlog format.
  - Fix query normalisation to not duplicate with bind queries.
    Normalisation of values are now tranformed into a single ? and no
    more 0 for numbers, two single quote for string. Thanks to vadv
    for the report.
  - Fix log level count. Thanks to Jean-Christophe Arnu for the report
  - Make pgbadger more compliant with B::Lint bare sub name.
  - Made perlcritic happy.
  - Add --prettify-json command line option to prettify JSON output.
    Default output is all in single line.
  - Fix Events distribution report.
  - Fix bug with --prefix when log_line_prefix contain multiple %%.
    Thanks to svb007 for the report.
  - Add --log-timezone  +/-XX  command line option to set the number
    of hours from GMT of the timezone that must be used to adjust
    date/time read from log file before beeing parsed. Using this
    option make more difficult log search with a date/time because the
    time will not be the same in the log. Note that you might still
    need to adjust the graph timezone using -Z when the client has not
    the same timezone. Thanks to xdexter for the feature request.
  - Remove INDEXES from the keyword list and add BUFFERS to this list.
  - Fix normalization of query using cursors.
  - Remove Dockerfile and documentation about docker run. pgBadger
    comes as a single Perl script without any dependence and it can
    be used on any plateform. It is a non sens to use docker to run
    pgbadger, if you don't want to install anything, just copy the
    file pgbadger where you want and execute it.
  - Fix broken grid when no temp files activity. Thanks to Pierre
    Giraud for the patch
  - Add doc warning about log_in_duration_statement vs log_duration +
    log_statement. Thanks to Julien Tachoire for the patch.
  - Apply timezone offset to bar charts. Thanks to Julien Tachoire
    for the patch.
  - Delete current temp file info if we meet an error for the same PID
    Thanks to Julien Tachoire for the patch.
  - Consistently use app= in examples, and support appname=
    Some of the usage examples used appname= in the prefix, but the
    code didn't recognize that token. Use app= in all examples, and
    add appname= to the prefix parser. Thanks to Christoph Berg for
    the patch
  - Fix wrong long name for option -J that should be --Jobs intead
    of --job_per_file. Thanks to Chad Trabant for the report and
    Etienne Bersac for the patch.
  - Ignore blib files. Thanks to Etienne Bersac for the patch.
  - Add consistency tests. Thanks to damien clochard for the patch.
  - doc update : stderr is not a default for -f. Thanks to Christophe
    Courtois for the patch.
  - Always update pod and README. Thanks to Etienne Bersac for
    the patch.
  - Add some regression tests. Thanks to Etienne Bersac for the patch.
  - Add editorconfig configuration. Thanks to Etienne Bersac for the
    patch.
  - Drop vi temp files from gitignore. Thanks to Etienne Bersac for
    the patch.
  - Add --include-time option to add the ability to choose times that
    you want to see, instead of excluding all the times you do not
    want to see. This is handy when wanting to view only one or two
    days from a week's worth of logs (simplifies down from multiple
    --exlucde-time options to one --include-time). Thanks to Wesley
    Bowman for the patch.
  - Check pod syntax. Thanks to Etienne Bersac for the patch.
  - Add HACKING to document tests. Thanks to Etienne Bersac for the
    patch.
  - Drop obsolete --bar-graph option. Thanks to Etienne Bersac for
    the patch.
  - Drop misleading .perltidyrc. This file date from 2012 and
    pgbadger code is far from compliant. perltidy unified diff is
    10k lines. Let's drop this. Thanks to Etienne Bersac for the
    patch.
  - Fix use of uninitialized value in SQL formatting. Thanks to John
    Krugger for the report and Jean-paul Argudo for the report.

@darold darold released this Jul 28, 2017 · 132 commits to master since this release

Assets 2

This release of pgBadger is a maintenance release that adds some new
features.

  • Add report of checkpoint distance and estimate.
  • Add support of AWS Redshift keywords to SQL code beautifier.
  • Add autodetection of log format in remote mode to allow remote parsing of pgbouncer log file together with PostgreSQL log file.

There's also some bugs fixes and features enhancements.

  - Fix reports with histogram that was not showing data upper than
    the last range.
  - Fix parsing of journalctl without the the log line number pattern
    ([%l-n]). Thanks to Christian Schmitt for the report.
  - Add report of checkpoint distance and estimate. Thanks to jjsantam
    for the feature request.
  - Append more information on what is done by script to update CSS
    and javascript files, tools/updt_embedded_rsc.pl.
  - Do not warn when all log files are empty and exit with code 0.
  - Fix build_log_line_prefix_regex() that does not include %n as a
    lookup in %regex_map. Thanks to ghosthound for the patch.
  - Change error level of "FATAL: cannot use CSV" to WARNING. Thanks
    to kong1man for the report.
  - Fix use of uninitialized value warning. Thanks to Payal for the
    report.
  - Add permission denied to error normalization
  - Update pgbadger to latest commit 5bdc018 of pgFormatter.
  - Add support for AWS Redshift keywords. Thanks to cavanaug for the
    feature request.
  - Fix missing query in temporary file report when the query was
    canceled. Thanks to Fabrizio de Royes Mello for the report.
  - Normalize query with binded parameters, replaced with a ?.
  - Sanity check to avoid end time before start time. Thanks to
    Christophe Courtois for the patch.
  - Fix a lot of mystyped words and do some grammatical fixes. Use
    'pgBadger' where it refers to the program and not the binary file.
    Also, use "official" expressions such as PgBouncer, GitHub, and
    CSS. POD file was synced with README. Thanks to Euler Taveira for
    the patch.
  - Menu is broken when --disable-type top_cancelled_info test and
    closing list must be inside disable_type test. While in it, ident
    disable_lock test. Thanks to Euler Taveira for the patch.
  - Fix use of uninitialized value. Thanks to johnkrugger for the
    report.
  - Remove test to read log file during log format auto-detection when
    the file is hosted remotly. Thanks to clomdd for the report.
  - Add autodetection of log format in remote mode to allow remote
    parsing of pgbouncer log file together with PostgreSQL log file.
  - Fix number of sessions wrongly increased after log line validation
    Thanks to Achilleas Mantzios for the report.
  - Minor reformatting of the pgBadger Description.
  - Fix repeated info in documentation. Thanks to cscatolini for the patch.

@darold darold released this Jan 24, 2017 · 163 commits to master since this release

Assets 2

2017-01-24 - v9.1

This release of pgBadger is a maintenance release that adds some new
features.

  • Add report of error class distribution when SQLState is available
    in the log_line_prefix (see %e placeholder).
  • Update SQL Beautifier to pgFormatter v1.6 code.
  • Improve error message normalization.
  • Add --normalized-only option to generate a text file containing all
    normalized queries found in a log with count.
  • Allow %c (session id) to replace %p (pid) as unique session id.
  • Add waiting for lock messages to event reports.
  • Add --start-monday option to start calendar weeks in Monday
    instead of default to Sunday.

There's also some bugs fixes and features enhancements.

  • Add report of error class distribution when SQLState is available
    in the log line prefix. Thanks to jacks33 for the feature request.
  • Fix incremental global index on resize. Thanks to clomdd for the
    report.
  • Fix command tag log_line_prefix placeholder %i to allow space
    character.
  • Fix --exclude-line options and removing of obsolete directory
    when retention is enabled and --noreport is used.
  • Fix typo in "vacuum activity table". Thanks to Nicolas Gollet for
    the patch.
  • Fix autovacuum report. Thanks to Nicolas Gollet for the patch.
  • Fix author of pgbadger's logo - Damien Cazeils and English in
    comments. Thanks to Thibaut Madelaine for the patch.
  • Add information about pgbouncer log format in the -f option.
    Thanks to clomdd for the report.
  • Add --normalized-only information in documentation.
  • Fix broken report of date-time introduced in previous patch.
  • Fix duration/query association when log_duration=on and
    log_statement=all. Thanks to Eric Jensen for the report.
  • Fix normalization of messages about advisory lock. Thanks to
    Thibaut Madelaine for the report.
  • Fix report of auto_explain output. Thanks to fch77700 for the
    report.
  • Fix unwanted log format auto detection with log entry from stdin.
    Thanks to Jesus Adolfo Parra for the report.
  • Add left open parentheses to the "stop" chars of regex to look
    for db client in the prefix to handle the PostgreSQL client
    string format that includes source port. Thanks to Jon Nelson
    for the patch.
  • Fix some spelling errors. Thanks to Jon Nelson for the patch.
  • Allow %c (session id) to replace %p (pid) as unique session id.
    Thanks to Jerryliuk for the report.
  • Allow pgbadger to parse default log_line_prefix that will be
    probably used in 10.0: '%m [%p] '
  • Fix missing first line with interpreter call.
  • Fix missing Avg values in CSV report. Thanks to Yosuke Tomita
    for the report.
  • Fix error message in autodetect_format() method.
  • Add --start-monday option to start calendar weeks in Monday
    instead of default to Sunday. Thanks to Joosep Mae for the feature
    request.
  • Fix --histo-average option. Thanks to Yves Martin for the report.
  • Remove plural form of --ssh-option in documentation. Thanks to
    mark-a-s for the report.
  • Fix --exclude-time filter and rewrite code to skip unwanted line
    as well code to update the progress bar. Thanks to Michael
    Chesterton for the report.
  • Fix support to %r placeholder in prefix instead of %h.

@darold darold released this Sep 2, 2016 · 204 commits to master since this release

Assets 2

2016-09-02 - v9.0

This major release of pgBadger is a port to bootstrap 3 and a version
upgrade of all resources files (CSS and Javascript). There's also some
bugs fixes and features enhancements.

Backward compatibility with old incremental report might be preserved.

  - Sources and licences of resources files are now on a dedicated
    subdirectory. A script to update their minified version embedded
    in pgbager script has been added. Thanks to Christoph Berg for
    the help and feature request.

  - Try to detect user/database/host from connection strings if
    log_connection is enabled and log_line_prefix doesn't include
    them.

    Extend the regex to autodetect database name, user name, client
    ip address and application name. The regex now are the following:

        db => qr/(?:db|database)=([^,]*)/;
        user => qr/(?:user|usr)=([^,]*)/;
        client => qr/(?:client|remote|ip|host)=([^,]*)/;
        appname => qr/(?:app|application)=([^,]*)/;

  - Add backward compatibility with older version of pgbadger in
    incremental mode by creating a subdirectory for new CSS and
    Javascript files. This subdirectory is named with the major
    version number of pgbadger.

  - Increase the size of the pgbadger logo that appears too small
    with the new font size.

  - Normalize detailed information in all reports.

  - Fix duplicate copy icon in locks report.

  - Fix missing chart on histogram of session time. Thanks to
    Guillaume Lelarge for the report.

  - Add LICENSE file noting the licenses used by the resource
    files. Thanks to Christoph Berg for the patch.

  - Add patch to jqplot library to fix an infinite loop when trying
    to download some charts. Thanks to Julien Tachoires for the help
    to solve this issue.

  - Script tools/updt_embedded_rsc.pl will apply the patch to resource
    file resources/jquery.jqplot.js and doesn't complain if it has
    already been applied.

  - Remove single last comma at end of pie chart dataset. Thanks to
    Julien Tachoires for the report.

  - Change display of normalized error

  - Remove unused or auto-generated files

  - Update all resources files (js+css) and create a directory to
    include source of javascript libraries used in pgbadger. There is
    also a new script tools/updt_embedded_rsc.pl the can be used to
    generate the minified version of those files and embedded them
    into pgbadger.  This script will also embedded the FontAwesome.otf
    open truetype font into the fontawesome.css file.

@darold darold released this Aug 27, 2016 · 222 commits to master since this release

Assets 2

2016-08-27 - v8.3

This is a maintenance release that fix some minor bugs. This release
also adds replication command messages statistics to the Events
reports.

  • Fix auto-detection of stderr format with timestamp as epoch (%n).
  • Fix histogram over multiples days to be cumulative per hour, not
    an average of the number of event per day.
  • Fix parsing of remote file that was failing when the file does
    not exists locally. Thanks to clomdd for the report.
  • Detect timezones like GMT+3 on CSV logs. Thanks to jacksonfoz
    for the patch.
  • Add replication command messages statistics to the Events
    reports. Thanks to Michael Paquier for the feature request.

This is the last minor version of the 8.x series, next major version
will include an upgrade of boostrap and jquery libraries which need
some major rewrite.

@darold darold released this Aug 11, 2016 · 229 commits to master since this release

Assets 2

2016-08-11 version 8.2

This is a maintenance release that fix some minor bug. There is also
some performances improvement up to 20% on huge files and some new
interesting features:

  • Multiprocessing can be used with pgbouncer log files.
  • pgBouncer and PostgreSQL log files can be used together in
    incremental mode.
  • With default or same prefix, stderr and syslog file can be
    parsed together, csvlog format can always be used.
  • Use a modal dialog window to download graphs as png images.
  • Add pl/pgSQL function information to queries when available.

See ChangeLog for the complete list of changes.

@darold darold released this Apr 21, 2016 · 258 commits to master since this release

Assets 2

2016-04-21 version 8.1

This is a maintenance release that fix a major issue introduced with
support to pgbouncer that prevent parsing of compressed PostgreSQL
log files and adds some improvements.

Here are the complete list of changes:

  • Fix one case where pid file remain after dying.
  • Add requirement of log_error_verbosity = default to documentation.
  • Report message "LOG: using stale statistics instead of current
    ones because stats collector is not responding" in events view.
  • Remove obsolete days when we are in binary mode with --noreport
  • Fix wrong report of statements responsible of temporary files.
    Thanks to Luan Nicolini Marcondes for the report. This patch also
    exclude line with log level LOCATION to be parsed.
  • Fix limit on number of sample at report generation and remove
    pending LAST_PARSED.tmp file.
  • Update load_stat() function and global variables to support
    pgbouncer statistics. Update version to 2.0.
  • Handle more kind or query types. Thanks to julien Rouhaud for
    the patch.
  • Fix pgbouncer log parser to handle message: FATAL: the database
    system is shutting down
  • Fix whitespace placed in between the E and the quote character.
    Thanks to clijunky for the report.
  • Fix a major issue introduced with support to pgbouncer that
    prevent parsing of compressed PostgreSQL log files. Thanks to
    Levente Birta for the report.

@darold darold released this Feb 22, 2016 · 274 commits to master since this release

Assets 2

This is a major release that adds support to pgbouncer log files. New pgbouncer reports are:

  • Request Throughput
  • Bytes I/O Throughput
  • Queries Average duration
  • Simultaneous sessions
  • Histogram of sessions times
  • Sessions per database
  • Sessions per user
  • Sessions per host
  • Established connections
  • Connections per database
  • Connections per user
  • Connections per host
  • Most used reserved pools
  • Most Frequent Errors/Events

pgbouncer log files can be parsed together with PostgreSQL logs.

It also adds a two new command line options:

  • --pgbouncer-only to only show pgbouncer related reports.
  • --rebuild to be able to rebuild all html reports in incremental output directory where binary data files are still available.

This release fixes a major bug introduced with journalctl code that was prevented the use of multiprocess feature.

@darold darold released this Jan 18, 2016 · 288 commits to master since this release

Assets 2

This is a maintenance release to fix a major bug that was breaking the incremental mode in pgBadger. It also adds some more reports and features.

  • Add --timezone=+/-HH to control the timezone used in charts. The
    javascript library runs at client side so the timezone used is
    the browser timezone so the displayed time in the charts can be
    different from the time in the log file.
  • Add /tmp/pgbadger.pid file to prevent cron jobs overlaping on
    same log files.
  • Add command line option --pid-dir to be able to run two pgbadger
    at the same time by setting an alternate path to the pid file.
  • Report information about "LOG: skipping analyze of ..." into
    events reports.
  • Report message "LOG: sending cancel to blocking autovacuum" into
    events reports. Useful to look for queries generating autovacuum
    kill on account of a lock issue.