Skip to content

Commit

Permalink
Add index scan to INSERT DML decompression
Browse files Browse the repository at this point in the history
In order to verify constraints, we have to decompress
batches that could contain duplicates of the tuples
we are inserting. To find such batches, we use heap
scans which can be very expensive if the compressed
chunk contains a lot of tuples. Doing an index scan
makes much more sense in this scenario and will
give great performance benefits.

Additionally, we don't want to create the decompressor
until we determine we actually want to decompress a
batch so we try to lazily initialize it once a batch
is found.
  • Loading branch information
antekresic committed Jun 20, 2024
1 parent eb3ef90 commit 4242fb0
Show file tree
Hide file tree
Showing 9 changed files with 700 additions and 445 deletions.
1 change: 1 addition & 0 deletions .unreleased/pr_7048
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implements: #7048 Add index scan to INSERT DML decompression
8 changes: 8 additions & 0 deletions src/nodes/chunk_dispatch/chunk_insert_state.c
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,7 @@ ts_chunk_insert_state_create(Oid chunk_relid, const ChunkDispatch *dispatch)
state->rel = rel;
state->result_relation_info = relinfo;
state->estate = dispatch->estate;
state->compressed_chunk_table_id = InvalidOid;
ts_set_compression_status(state, chunk);

if (relinfo->ri_RelationDesc->rd_rel->relhasindex && relinfo->ri_IndexRelationDescs == NULL)
Expand Down Expand Up @@ -634,7 +635,14 @@ ts_set_compression_status(ChunkInsertState *state, const Chunk *chunk)
{
state->chunk_compressed = ts_chunk_is_compressed(chunk);
if (state->chunk_compressed)
{
state->chunk_partial = ts_chunk_is_partial(chunk);
if (!OidIsValid(state->compressed_chunk_table_id))
{
Chunk *comp = ts_chunk_get_by_id(chunk->fd.compressed_chunk_id, true);
state->compressed_chunk_table_id = comp->table_id;
}
}
}

extern void
Expand Down
2 changes: 2 additions & 0 deletions src/nodes/chunk_dispatch/chunk_insert_state.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ typedef struct ChunkInsertState
/* for tracking compressed chunks */
bool chunk_compressed;
bool chunk_partial;

Oid compressed_chunk_table_id;
} ChunkInsertState;

typedef struct ChunkDispatch ChunkDispatch;
Expand Down
Loading

0 comments on commit 4242fb0

Please sign in to comment.