duplicate gp_fastsequence value (ctid) appears in ao table

[ppggff]

Bug Report

Greenplum version or build

master 59eb923

OS version and uname -a

Linux 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 30 15:51:32 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

autoconf options used ( config.status --config )

Installation information ( pg_config )

Expected behavior

no duplicate gp_fastsequence value (ctid) appears in ao table

Actual behavior

duplicate gp_fastsequence value (ctid) appears in ao table

Step to reproduce the behavior

begin;
create table a(a int, b int) with (appendonly = true) distributed by (a);
--create index ia on a(b);
insert into a values(1,1);
insert into a values(1,1);
insert into a values(1,1);
savepoint s1;
truncate a;
insert into a values(1,1);
rollback to s1;
insert into a values(1,2);

postgres=# select ctid,* from a;
  ctid   | a | b
---------+---+---
 (0,2)   | 1 | 1
 (0,102) | 1 | 1
 (0,202) | 1 | 1
 (0,102) | 1 | 2
(4 rows)

panic with index

begin;
create table a(a int, b int) with (appendonly = true) distributed by (a);
create index ia on a(b);
insert into a values(1,1);
insert into a values(1,1);
insert into a values(1,1);
savepoint s1;
truncate a;
insert into a values(1,1);
rollback to s1;
insert into a values(1,2);

ERROR:  Unexpected internal error (assert.c:44)  (seg1 127.0.1.1:7003 pid=22234) (assert.c:44)
DETAIL:  FailedAssertion("!(entry->firstRowNum < firstRowNum)", File: "appendonlyblockdirectory.c", Line: 747)

[SmartKeyerror]

RCA:

If we run TRUNCATE TABLE in subtransaction, it will not delete all data directly, for it's can't roll back this sub-transaction, so Greenplum will create a new empty storage file for the relation, and assign it as the relfilenode value. The old storage file is scheduled for deletion at commit.

But for AO table, the relfilenode should not be changed to gp_fastsequence, since AO table relies on the last_sequence to generate a new firstRowNumber to a new file block.

firstRowNumber should be guaranteed to keep incrementing in all cases, but this failed at the truncate table in subtransaction.

[ashwinstar]

Thanks for reporting. The problem is worst for 6X_STABLE as truncate even outside create table will produce duplicate CTID and hence incorrect indexes.

drop table if exists ao;
DROP TABLE
create table ao(a int, b int) with (appendonly=true) distributed by (a);
CREATE TABLE
--create index idx on ao(b);
insert into ao values(1,1);
INSERT 0 1
select gp_segment_id, xmin, xmax, * from gp_dist_random('gp_fastsequence') where gp_segment_id=1 and objid in (select segrelid from pg_appendonly where relid = 'ao'::regclass);
 gp_segment_id | xmin | xmax | objid | objmod | last_sequence 
---------------+------+------+-------+--------+---------------
             1 |  732 |    0 | 24599 |      1 |           100
             1 |  732 |    0 | 24599 |      0 |             0
(2 rows)

insert into ao values(1,1);
INSERT 0 1
insert into ao values(1,1);
INSERT 0 1
select gp_segment_id, xmin, xmax, * from gp_dist_random('gp_fastsequence') where gp_segment_id=1 and objid in (select segrelid from pg_appendonly where relid = 'ao'::regclass);
 gp_segment_id | xmin | xmax | objid | objmod | last_sequence 
---------------+------+------+-------+--------+---------------
             1 |  732 |    0 | 24599 |      1 |           300
             1 |  732 |    0 | 24599 |      0 |             0
(2 rows)

select ctid, * from ao;
      ctid      | a | b 
----------------+---+---
 (33554432,2)   | 1 | 1
 (33554432,102) | 1 | 1
 (33554432,202) | 1 | 1
(3 rows)

begin;
BEGIN
truncate table ao;
TRUNCATE TABLE
insert into ao values(1,2);
INSERT 0 1
select gp_segment_id, xmin, xmax, * from gp_dist_random('gp_fastsequence') where gp_segment_id=1 and objid in (select segrelid from pg_appendonly where relid = 'ao'::regclass);
 gp_segment_id | xmin | xmax | objid | objmod | last_sequence 
---------------+------+------+-------+--------+---------------
             1 |  732 |    0 | 24599 |      1 |           100
             1 |  732 |    0 | 24599 |      0 |             0
(2 rows)

abort;
ROLLBACK
select gp_segment_id, xmin, xmax from gp_dist_random('gp_fastsequence') where gp_segment_id=1 and objid in (select segrelid from pg_appendonly where relid = 'ao'::regclass);
 gp_segment_id | xmin | xmax 
---------------+------+------
             1 |  732 |    0
             1 |  732 |    0
(2 rows)

insert into ao values(1,3);
INSERT 0 1
select gp_segment_id, xmin, xmax, * from gp_dist_random('gp_fastsequence') where gp_segment_id=1 and objid in (select segrelid from pg_appendonly where relid = 'ao'::regclass);
 gp_segment_id | xmin | xmax | objid | objmod | last_sequence 
---------------+------+------+-------+--------+---------------
             1 |  732 |    0 | 24599 |      1 |           200
             1 |  732 |    0 | 24599 |      0 |             0
(2 rows)

select ctid, gp_segment_id,* from ao order by ctid;
      ctid      | gp_segment_id | a | b 
----------------+---------------+---+---
 (33554432,2)   |             1 | 1 | 1
 (33554432,102) |             1 | 1 | 3  <<====================
 (33554432,102) |             1 | 1 | 1
 (33554432,202) |             1 | 1 | 1
(4 rows)

hence the fix in #13762 becomes much more important.

The reason its more serious for 6X_STABLE as in master branch inserts after truncate is using only segfile 0 due to code in choose_segno_internal() and more importantly ShouldUseReservedSegno() but that's not the case for 6X_STABLE and doesn't need to be. It's just due to ShouldUseReservedSegno() making decision based on xmin of pg_class for table and truncate modifies it and hence results in picking segfile 0.