Skip to content

Commit

Permalink
Add new pg_walsummary tool.
Browse files Browse the repository at this point in the history
This can dump the contents of the WAL summary files found in
pg_wal/summaries. Normally, this shouldn't really be something anyone
needs to do, but it may be needed for debugging problems with
incremental backup, or could possibly be useful to external tools.

Discussion: http://postgr.es/m/CA+Tgmobvqqj-DW9F7uUzT-cQqs6wcVb-Xhs=w=hzJnXSE-kRGw@mail.gmail.com
  • Loading branch information
robertmhaas committed Jan 11, 2024
1 parent d9ef650 commit ee1bfd1
Show file tree
Hide file tree
Showing 13 changed files with 600 additions and 0 deletions.
1 change: 1 addition & 0 deletions doc/src/sgml/ref/allfiles.sgml
Expand Up @@ -219,6 +219,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgtesttiming SYSTEM "pgtesttiming.sgml">
<!ENTITY pgupgrade SYSTEM "pgupgrade.sgml">
<!ENTITY pgwaldump SYSTEM "pg_waldump.sgml">
<!ENTITY pgwalsummary SYSTEM "pg_walsummary.sgml">
<!ENTITY postgres SYSTEM "postgres-ref.sgml">
<!ENTITY psqlRef SYSTEM "psql-ref.sgml">
<!ENTITY reindexdb SYSTEM "reindexdb.sgml">
Expand Down
122 changes: 122 additions & 0 deletions doc/src/sgml/ref/pg_walsummary.sgml
@@ -0,0 +1,122 @@
<!--
doc/src/sgml/ref/pg_walsummary.sgml
PostgreSQL documentation
-->

<refentry id="app-pgwalsummary">
<indexterm zone="app-pgwalsummary">
<primary>pg_walsummary</primary>
</indexterm>

<refmeta>
<refentrytitle><application>pg_walsummary</application></refentrytitle>
<manvolnum>1</manvolnum>
<refmiscinfo>Application</refmiscinfo>
</refmeta>

<refnamediv>
<refname>pg_walsummary</refname>
<refpurpose>print contents of WAL summary files</refpurpose>
</refnamediv>

<refsynopsisdiv>
<cmdsynopsis>
<command>pg_walsummary</command>
<arg rep="repeat" choice="opt"><replaceable>option</replaceable></arg>
<arg rep="repeat"><replaceable>file</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>

<refsect1>
<title>Description</title>
<para>
<application>pg_walsummary</application> is used to print the contents of
WAL summary files. These binary files are found with the
<literal>pg_wal/summaries</literal> subdirectory of the data directory,
and can be converted to text using this tool. This is not ordinarily
necessary, since WAL summary files primarily exist to support
<link linkend="backup-incremental-backup">incremental backup</link>,
but it may be useful for debugging purposes.
</para>

<para>
A WAL summary file is indexed by tablespace OID, relation OID, and relation
fork. For each relation fork, it stores the list of blocks that were
modified by WAL within the range summarized in the file. It can also
store a "limit block," which is 0 if the relation fork was created or
truncated within the relevant WAL range, and otherwise the shortest length
to which the relation fork was truncated. If the relation fork was not
created, deleted, or truncated within the relevant WAL range, the limit
block is undefined or infinite and will not be printed by this tool.
</para>
</refsect1>

<refsect1>
<title>Options</title>

<para>
<variablelist>
<varlistentry>
<term><option>-i</option></term>
<term><option>--indivudual</option></term>
<listitem>
<para>
By default, <literal>pg_walsummary</literal> prints one line of output
for each range of one or more consecutive modified blocks. This can
make the output a lot briefer, since a relation where all blocks from
0 through 999 were modified will produce only one line of output rather
than 1000 separate lines. This option requests a separate line of
output for every modified block.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term><option>-q</option></term>
<term><option>--quiet</option></term>
<listitem>
<para>
Do not print any output, except for errors. This can be useful
when you want to know whether a WAL summary file can be successfully
parsed but don't care about the contents.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
<listitem>
<para>
Shows help about <application>pg_walsummary</application> command line
arguments, and exits.
</para>
</listitem>
</varlistentry>

</variablelist>
</para>

</refsect1>

<refsect1>
<title>Environment</title>

<para>
The environment variable <envar>PG_COLOR</envar> specifies whether to use
color in diagnostic messages. Possible values are
<literal>always</literal>, <literal>auto</literal> and
<literal>never</literal>.
</para>
</refsect1>

<refsect1>
<title>See Also</title>

<simplelist type="inline">
<member><xref linkend="app-pgbasebackup"/></member>
<member><xref linkend="app-pgcombinebackup"/></member>
</simplelist>
</refsect1>

</refentry>
1 change: 1 addition & 0 deletions doc/src/sgml/reference.sgml
Expand Up @@ -289,6 +289,7 @@
&pgtesttiming;
&pgupgrade;
&pgwaldump;
&pgwalsummary;
&postgres;

</reference>
Expand Down
1 change: 1 addition & 0 deletions src/bin/Makefile
Expand Up @@ -31,6 +31,7 @@ SUBDIRS = \
pg_upgrade \
pg_verifybackup \
pg_waldump \
pg_walsummary \
pgbench \
psql \
scripts
Expand Down
1 change: 1 addition & 0 deletions src/bin/meson.build
Expand Up @@ -17,6 +17,7 @@ subdir('pg_test_timing')
subdir('pg_upgrade')
subdir('pg_verifybackup')
subdir('pg_waldump')
subdir('pg_walsummary')
subdir('pgbench')
subdir('pgevent')
subdir('psql')
Expand Down
1 change: 1 addition & 0 deletions src/bin/pg_walsummary/.gitignore
@@ -0,0 +1 @@
pg_walsummary
48 changes: 48 additions & 0 deletions src/bin/pg_walsummary/Makefile
@@ -0,0 +1,48 @@
#-------------------------------------------------------------------------
#
# Makefile for src/bin/pg_walsummary
#
# Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
# Portions Copyright (c) 1994, Regents of the University of California
#
# src/bin/pg_walsummary/Makefile
#
#-------------------------------------------------------------------------

PGFILEDESC = "pg_walsummary - print contents of WAL summary files"
PGAPPICON=win32

subdir = src/bin/pg_walsummary
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global

override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils

OBJS = \
$(WIN32RES) \
pg_walsummary.o

all: pg_walsummary

pg_walsummary: $(OBJS) | submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)


install: all installdirs
$(INSTALL_PROGRAM) pg_walsummary$(X) '$(DESTDIR)$(bindir)/pg_walsummary$(X)'

installdirs:
$(MKDIR_P) '$(DESTDIR)$(bindir)'

uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_walsummary$(X)'

clean distclean maintainer-clean:
rm -f pg_walsummary$(X) $(OBJS)

check:
$(prove_check)

installcheck:
$(prove_installcheck)
30 changes: 30 additions & 0 deletions src/bin/pg_walsummary/meson.build
@@ -0,0 +1,30 @@
# Copyright (c) 2022-2023, PostgreSQL Global Development Group

pg_walsummary_sources = files(
'pg_walsummary.c',
)

if host_system == 'windows'
pg_walsummary_sources += rc_bin_gen.process(win32ver_rc, extra_args: [
'--NAME', 'pg_walsummary',
'--FILEDESC', 'pg_walsummary - print contents of WAL summary files',])
endif

pg_walsummary = executable('pg_walsummary',
pg_walsummary_sources,
dependencies: [frontend_code],
kwargs: default_bin_args,
)
bin_targets += pg_walsummary

tests += {
'name': 'pg_walsummary',
'sd': meson.current_source_dir(),
'bd': meson.current_build_dir(),
'tap': {
'tests': [
't/001_basic.pl',
't/002_blocks.pl',
],
}
}
6 changes: 6 additions & 0 deletions src/bin/pg_walsummary/nls.mk
@@ -0,0 +1,6 @@
# src/bin/pg_combinebackup/nls.mk
CATALOG_NAME = pg_walsummary
GETTEXT_FILES = $(FRONTEND_COMMON_GETTEXT_FILES) \
pg_walsummary.c
GETTEXT_TRIGGERS = $(FRONTEND_COMMON_GETTEXT_TRIGGERS)
GETTEXT_FLAGS = $(FRONTEND_COMMON_GETTEXT_FLAGS)

0 comments on commit ee1bfd1

Please sign in to comment.