Skip to content

Commit

Permalink
Enable insert multiple file on AO tables.
Browse files Browse the repository at this point in the history
Changes:

When enable_parallel is on, the insertMultiFiles flag is set to true when:
gp_appendonly_insert_files GUC value is greater than 1 (indicating multiple files used)
and the insert is not reserving a specific segment.
When insertMultiFiles is true, data will be appended to multiple file segments in parallel.
The default value of gp_appendonly_insert_files GUC is increased from 0 to 4, to enable up to 4 parallel insert files by default.
This allows append-only tables to scale insert performance by writing to multiple file segments in parallel. The number of parallel segments can be configured using the gp_appendonly_insert_files GUC.
  • Loading branch information
yjhjstz authored and yjhjstz committed Jun 30, 2023
1 parent 824f4e8 commit 5acbf4e
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 3 deletions.
4 changes: 3 additions & 1 deletion src/backend/access/aocs/aocsam.c
Original file line number Diff line number Diff line change
Expand Up @@ -1068,7 +1068,9 @@ aocs_insert_init(Relation rel, int segno)
rel, segno, tupleDesc->natts, true);

/* should not enable insertMultiFiles if the table is created by own transaction */
desc->insertMultiFiles = ((gp_appendonly_insert_files > 1) && !ShouldUseReservedSegno(rel, CHOOSE_MODE_WRITE));
desc->insertMultiFiles = enable_parallel &&
gp_appendonly_insert_files > 1 &&
!ShouldUseReservedSegno(rel, CHOOSE_MODE_WRITE);
return desc;
}

Expand Down
4 changes: 3 additions & 1 deletion src/backend/access/appendonly/appendonlyam.c
Original file line number Diff line number Diff line change
Expand Up @@ -2852,7 +2852,9 @@ aoInsertDesc->appendOnlyMetaDataSnapshot, //CONCERN:Safe to assume all block dir
rel, segno, 1, false);

/* should not enable insertMultiFiles if the table is created by own transaction */
aoInsertDesc->insertMultiFiles = ((gp_appendonly_insert_files > 1) && !ShouldUseReservedSegno(rel, CHOOSE_MODE_WRITE));
aoInsertDesc->insertMultiFiles = enable_parallel &&
gp_appendonly_insert_files > 1 &&
!ShouldUseReservedSegno(rel, CHOOSE_MODE_WRITE);
return aoInsertDesc;
}

Expand Down
2 changes: 1 addition & 1 deletion src/backend/utils/misc/guc_gp.c
Original file line number Diff line number Diff line change
Expand Up @@ -3165,7 +3165,7 @@ struct config_int ConfigureNamesInt_gp[] =
NULL
},
&gp_appendonly_insert_files,
0, 0, 127,
4, 0, 127,
NULL, NULL, NULL
},

Expand Down

0 comments on commit 5acbf4e

Please sign in to comment.