Skip to content

Commit 2a5f853

Browse files
jeff-davishari90
jeff-davis
authored and
hari90
committed
Transfer statistics during pg_upgrade.
yb conflict resolutions: - doc/src/sgml/ref/pg_dump.sgml Only changes that were not conflicting were picked. - src/bin/pg_dump/pg_backup_archiver.c >@@ -1036,15 +1041,21 @@ restore_toc_entry(ArchiveHandle *AH, TocEntry *te, bool is_parallel) <@@ -966,10 +972,17 @@ restore_toc_entry(ArchiveHandle *AH, TocEntry *te, bool is_parallel) >- if ((reqs & (REQ_SCHEMA | REQ_DATA)) != 0 && ropt->txn_size > 0) >+ if ((reqs & (REQ_SCHEMA | REQ_DATA | REQ_STATS)) != 0 && ropt->txn_size > 0) Cause: Missing 959b38d: if conditions related to `ropt->txn_size` was added. Resolution: Skip the incoming change since it is not relevant to pg15. - src/bin/pg_dump/pg_dump.h >@@ -83,10 +83,13 @@ typedef enum <@@ -82,9 +82,12 @@ typedef enum >+#define NUM_DUMPABLE_OBJECT_TYPES (DO_SUBSCRIPTION_REL + 1) <+#define NUM_DUMPABLE_OBJECT_TYPES (DO_SUBSCRIPTION + 1) Cause: Missing 9a17be1: DO_SUBSCRIPTION_REL was added as the last element in the DumpableObjectType macro. Resolution: Set NUM_DUMPABLE_OBJECT_TYPES to DO_SUBSCRIPTION which is the last element in DumpableObjectType. - src/bin/pg_dump/pg_dump_sort.c >@@ -148,11 +149,12 @@ static const int dbObjectTypePriority[] = >+ [DO_REL_STATS] = PRIO_STATISTICS_DATA_DATA, Cause: Missing afd8ef3: CdbObjectTypePriority array switched to c99-designated initializer syntax. Resolution: Use pg15 style syntax to assign PRIO_STATISTICS_DATA_DATA. StaticAssertDecl conflict is the same as the one mentioned in pg_dump.h. <@@ -146,10 +147,11 @@ static const int dbObjectTypePriority[] = <+ PRIO_STATISTICS_DATA_DATA, /* DO_STATISTICS_DATA_DATA */ >-StaticAssertDecl(lengthof(dbObjectTypePriority) == (DO_SUBSCRIPTION_REL + 1), <-StaticAssertDecl(lengthof(dbObjectTypePriority) == (DO_SUBSCRIPTION + 1), Cause: Missing <commit id>: <Explanation of differences> Resolution: <Explanation of resolution> - src/bin/pg_dump/t/002_pg_dump.pl >@@ -1432,6 +1464,7 @@ my %tests = ( >+ no_schema => 1, Cause: Missing 00d9dcf: 'LO create (with no data)' test was added. Resolution: Skip test since its not in pg15. >@@ -3310,6 +3354,7 @@ my %tests = ( >+ no_schema => 1, Cause: Missing 1a05c1d: 'COPY test_compression_method' test was added. Resolution: Skip test since its not in pg15. >@@ -3480,6 +3525,7 @@ my %tests = ( >+ no_schema => 1, Cause: Missing a563c24: 'COPY measurement' test was added. Resolution: Skip test since its not in pg15. - src/test/recovery/t/027_stream_regress.pl >@@ -107,7 +107,7 @@ command_ok( <@@ -93,14 +93,14 @@ $node_primary->wait_for_catchup($node_standby_1, 'replay', >- '--no-sync', >+ '--no-sync', '--no-statistics', <- '--no-sync', '-p', $node_primary->port, <+ '--no-sync', '--no-statistics', '-p', $node_primary->port, Cause: Missing ce1b0f9: TAP test improvements were made. Resolution: Combine merge the new --no-statistics param with existing pg15 params. >@@ -116,7 +116,7 @@ command_ok( >- '--no-sync', >+ '--no-sync', '--no-statistics', <- '--no-sync', '-p', $node_standby_1->port <+ '--no-sync', '--no-statistics', '-p', $node_standby_1->port Cause: Missing ce1b0f9: TAP test improvements were made. Resolution: Combine merge the new --no-statistics param with existing pg15 params. Add support to pg_dump for dumping stats, and use that during pg_upgrade so that statistics are transferred during upgrade. In most cases this removes the need for a costly re-analyze after upgrade. Some statistics are not transferred, such as extended statistics or statistics with a custom stakind. Now pg_dump accepts the options --schema-only, --no-schema, --data-only, --no-data, --statistics-only, and --no-statistics; which allow all combinations of schema, data, and/or stats. The options are named this way to preserve compatibility with the previous --schema-only and --data-only options. Statistics are in SECTION_DATA, unless the object itself is in SECTION_POST_DATA. The stats are represented as calls to pg_restore_relation_stats() and pg_restore_attribute_stats(). Author: Corey Huinker, Jeff Davis Reviewed-by: Jian He Discussion: https://postgr.es/m/CADkLM=fzX7QX6r78fShWDjNN3Vcr4PVAnvXxQ4DiGy6V=0bCUA@mail.gmail.com Discussion: https://postgr.es/m/CADkLM%3DcB0rF3p_FuWRTMSV0983ihTRpsH%2BOCpNyiqE7Wk0vUWA%40mail.gmail.com (cherry picked from commit 1fd1bd8)
1 parent e9c1b29 commit 2a5f853

22 files changed

+826
-67
lines changed

‎doc/src/sgml/ref/pg_dump.sgml

Lines changed: 31 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ PostgreSQL documentation
119119
<term><option>--data-only</option></term>
120120
<listitem>
121121
<para>
122-
Dump only the data, not the schema (data definitions).
122+
Dump only the data, not the schema (data definitions) or statistics.
123123
Table data, large objects, and sequence values are dumped.
124124
</para>
125125

@@ -508,10 +508,11 @@ PostgreSQL documentation
508508
<term><option>--schema-only</option></term>
509509
<listitem>
510510
<para>
511-
Dump only the object definitions (schema), not data.
511+
Dump only the object definitions (schema), not data or statistics.
512512
</para>
513513
<para>
514-
This option is the inverse of <option>--data-only</option>.
514+
This option is mutually exclusive to <option>--data-only</option>
515+
and <option>--statistics-only</option>.
515516
It is similar to, but for historical reasons not identical to,
516517
specifying
517518
<option>--section=pre-data --section=post-data</option>.
@@ -706,7 +707,8 @@ PostgreSQL documentation
706707
<term><option>--disable-triggers</option></term>
707708
<listitem>
708709
<para>
709-
This option is relevant only when creating a data-only dump.
710+
This option is relevant only when creating a dump that includes data
711+
but does not include schema.
710712
It instructs <application>pg_dump</application> to include commands
711713
to temporarily disable triggers on the target tables while
712714
the data is restored. Use this if you have referential
@@ -764,7 +766,8 @@ PostgreSQL documentation
764766
though you do not need the data in it.
765767
</para>
766768
<para>
767-
To exclude data for all tables in the database, see <option>--schema-only</option>.
769+
To exclude data for all tables in the database, see <option>--schema-only</option>
770+
or <option>--statistics-only</option>.
768771
</para>
769772
</listitem>
770773
</varlistentry>
@@ -909,6 +912,24 @@ PostgreSQL documentation
909912
</listitem>
910913
</varlistentry>
911914

915+
<varlistentry>
916+
<term><option>--no-schema</option></term>
917+
<listitem>
918+
<para>
919+
Do not dump schema (data definitions).
920+
</para>
921+
</listitem>
922+
</varlistentry>
923+
924+
<varlistentry>
925+
<term><option>--no-statistics</option></term>
926+
<listitem>
927+
<para>
928+
Do not dump statistics.
929+
</para>
930+
</listitem>
931+
</varlistentry>
932+
912933
<varlistentry>
913934
<term><option>--no-subscriptions</option></term>
914935
<listitem>
@@ -1047,9 +1068,11 @@ PostgreSQL documentation
10471068
</para>
10481069
<para>
10491070
The data section contains actual table data, large-object
1050-
contents, and sequence values.
1071+
contents, statitistics for tables and materialized views and
1072+
sequence values.
10511073
Post-data items include definitions of indexes, triggers, rules,
1052-
and constraints other than validated check constraints.
1074+
statistics for indexes, and constraints other than validated check
1075+
constraints.
10531076
Pre-data items include all other data definition items.
10541077
</para>
10551078
</listitem>
@@ -1355,7 +1378,7 @@ CREATE DATABASE foo WITH TEMPLATE template0;
13551378
</para>
13561379

13571380
<para>
1358-
When a data-only dump is chosen and the option <option>--disable-triggers</option>
1381+
When a dump without schema is chosen and the option <option>--disable-triggers</option>
13591382
is used, <application>pg_dump</application> emits commands
13601383
to disable triggers on user tables before inserting the data,
13611384
and then commands to re-enable them after the data has been

‎doc/src/sgml/ref/pg_dumpall.sgml

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ PostgreSQL documentation
8181
<term><option>--data-only</option></term>
8282
<listitem>
8383
<para>
84-
Dump only the data, not the schema (data definitions).
84+
Dump only the data, not the schema (data definitions) or statistics.
8585
</para>
8686
</listitem>
8787
</varlistentry>
@@ -234,6 +234,16 @@ PostgreSQL documentation
234234
</listitem>
235235
</varlistentry>
236236

237+
<varlistentry>
238+
<term><option>--statistics-only</option></term>
239+
<listitem>
240+
<para>
241+
Dump only the statistics, not the schema (data definitions) or data.
242+
Statistics for tables, materialized views, and indexes are dumped.
243+
</para>
244+
</listitem>
245+
</varlistentry>
246+
237247
<varlistentry>
238248
<term><option>--binary-upgrade</option></term>
239249
<listitem>
@@ -276,7 +286,7 @@ PostgreSQL documentation
276286
<term><option>--disable-triggers</option></term>
277287
<listitem>
278288
<para>
279-
This option is relevant only when creating a data-only dump.
289+
This option is relevant only when creating a dump with data and without schema.
280290
It instructs <application>pg_dumpall</application> to include commands
281291
to temporarily disable triggers on the target tables while
282292
the data is restored. Use this if you have referential
@@ -391,6 +401,15 @@ PostgreSQL documentation
391401
</listitem>
392402
</varlistentry>
393403

404+
<varlistentry>
405+
<term><option>--no-data</option></term>
406+
<listitem>
407+
<para>
408+
Do not dump data.
409+
</para>
410+
</listitem>
411+
</varlistentry>
412+
394413
<varlistentry>
395414
<term><option>--no-publications</option></term>
396415
<listitem>
@@ -416,6 +435,15 @@ PostgreSQL documentation
416435
</listitem>
417436
</varlistentry>
418437

438+
<varlistentry>
439+
<term><option>--no-schema</option></term>
440+
<listitem>
441+
<para>
442+
Do not dump schema (data definitions).
443+
</para>
444+
</listitem>
445+
</varlistentry>
446+
419447
<varlistentry>
420448
<term><option>--no-security-labels</option></term>
421449
<listitem>
@@ -425,6 +453,15 @@ PostgreSQL documentation
425453
</listitem>
426454
</varlistentry>
427455

456+
<varlistentry>
457+
<term><option>--no-statistics</option></term>
458+
<listitem>
459+
<para>
460+
Do not dump statistics.
461+
</para>
462+
</listitem>
463+
</varlistentry>
464+
428465
<varlistentry>
429466
<term><option>--no-subscriptions</option></term>
430467
<listitem>

‎doc/src/sgml/ref/pg_restore.sgml

Lines changed: 44 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ PostgreSQL documentation
9494
<term><option>--data-only</option></term>
9595
<listitem>
9696
<para>
97-
Restore only the data, not the schema (data definitions).
97+
Restore only the data, not the schema (data definitions) or statistics.
9898
Table data, large objects, and sequence values are restored,
9999
if present in the archive.
100100
</para>
@@ -403,10 +403,11 @@ PostgreSQL documentation
403403
to the extent that schema entries are present in the archive.
404404
</para>
405405
<para>
406-
This option is the inverse of <option>--data-only</option>.
406+
This option is mutually exclusive of <option>--data-only</option>
407+
and <option>--statistics-only</option>.
407408
It is similar to, but for historical reasons not identical to,
408409
specifying
409-
<option>--section=pre-data --section=post-data</option>.
410+
<option>--section=pre-data --section=post-data --no-statistics</option>.
410411
</para>
411412
<para>
412413
(Do not confuse this with the <option>--schema</option> option, which
@@ -519,6 +520,15 @@ PostgreSQL documentation
519520
</listitem>
520521
</varlistentry>
521522

523+
<varlistentry>
524+
<term><option>--statistics-only</option></term>
525+
<listitem>
526+
<para>
527+
Restore only the statistics, not schema (data definitions) or data.
528+
</para>
529+
</listitem>
530+
</varlistentry>
531+
522532
<varlistentry>
523533
<term><option>-1</option></term>
524534
<term><option>--single-transaction</option></term>
@@ -537,7 +547,7 @@ PostgreSQL documentation
537547
<term><option>--disable-triggers</option></term>
538548
<listitem>
539549
<para>
540-
This option is relevant only when performing a data-only restore.
550+
This option is relevant only when performing a restore without schema.
541551
It instructs <application>pg_restore</application> to execute commands
542552
to temporarily disable triggers on the target tables while
543553
the data is restored. Use this if you have referential
@@ -601,6 +611,16 @@ PostgreSQL documentation
601611
</listitem>
602612
</varlistentry>
603613

614+
<varlistentry>
615+
<term><option>--no-data</option></term>
616+
<listitem>
617+
<para>
618+
Do not output commands to restore data, even if the archive
619+
contains them.
620+
</para>
621+
</listitem>
622+
</varlistentry>
623+
604624
<varlistentry>
605625
<term><option>--no-data-for-failed-tables</option></term>
606626
<listitem>
@@ -633,6 +653,16 @@ PostgreSQL documentation
633653
</listitem>
634654
</varlistentry>
635655

656+
<varlistentry>
657+
<term><option>--no-schema</option></term>
658+
<listitem>
659+
<para>
660+
Do not output commands to restore schema (data definitions), even if
661+
the archive contains them.
662+
</para>
663+
</listitem>
664+
</varlistentry>
665+
636666
<varlistentry>
637667
<term><option>--no-security-labels</option></term>
638668
<listitem>
@@ -643,6 +673,16 @@ PostgreSQL documentation
643673
</listitem>
644674
</varlistentry>
645675

676+
<varlistentry>
677+
<term><option>--no-statistics</option></term>
678+
<listitem>
679+
<para>
680+
Do not output commands to restore statistics, even if the archive
681+
contains them.
682+
</para>
683+
</listitem>
684+
</varlistentry>
685+
646686
<varlistentry>
647687
<term><option>--no-subscriptions</option></term>
648688
<listitem>

‎doc/src/sgml/ref/pgupgrade.sgml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,15 @@ PostgreSQL documentation
145145
</listitem>
146146
</varlistentry>
147147

148+
<varlistentry>
149+
<term><option>--no-statistics</option></term>
150+
<listitem>
151+
<para>
152+
Do not restore statistics from the old cluster into the new cluster.
153+
</para>
154+
</listitem>
155+
</varlistentry>
156+
148157
<varlistentry>
149158
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
150159
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>

‎src/bin/pg_dump/pg_backup.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,7 @@ typedef struct _restoreOptions
154154
/* flags derived from the user-settable flags */
155155
bool dumpSchema;
156156
bool dumpData;
157+
bool dumpStatistics;
157158
} RestoreOptions;
158159

159160
typedef struct _dumpOptions
@@ -202,6 +203,7 @@ typedef struct _dumpOptions
202203
/* flags derived from the user-settable flags */
203204
bool dumpSchema;
204205
bool dumpData;
206+
bool dumpStatistics;
205207
} DumpOptions;
206208

207209
/*

0 commit comments

Comments
 (0)