Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PANIC if prefetch innerplan refers outerParams #9679

Closed
kainwen opened this issue Mar 4, 2020 · 1 comment
Closed

PANIC if prefetch innerplan refers outerParams #9679

kainwen opened this issue Mar 4, 2020 · 1 comment
Assignees
Labels

Comments

@kainwen
Copy link
Member

@kainwen kainwen commented Mar 4, 2020

create table a (i int4);
create table b (i int4);
create table c (i int4, j int4);

insert into a select g from generate_series(1,1) g;
insert into b select g from generate_series(1,1) g;
insert into c select g, g from generate_series(1, 100) g;

create index on c (i,j);

-- In order to get the plan we want, Index Scan on 'c' must appear
-- much cheaper than a Seq Scan. In order to keep this test quick and small,
-- we don't want to actually create a huge table, so cheat a little and
-- force that stats to make it look big to the planner.
set allow_system_table_mods = on;
update pg_class set reltuples=10000000 where oid ='c'::regclass;

set enable_hashjoin=off;
set enable_mergejoin=off;
set enable_nestloop=on;

-- the plan should look something like this:
--
--                                 QUERY PLAN
-- ---------------------------------------------------------------------------
--  Gather Motion 3:1  (slice1; segments: 3)
--    ->  Nested Loop [1]
--          ->  Broadcast Motion 3:3  (slice2; segments: 3)
--                ->  Seq Scan on b
--          ->  Materialize  [6]
--                ->  Nested Loop [2]
--                      Join Filter: (b.i = a.i)
--                      ->  Materialize [5]
--                            ->  Broadcast Motion 3:3  (slice3; segments: 3) [3]
--                                  ->  Seq Scan on a
--                      ->  Index Only Scan using c_i_j_idx on c
--                            Index Cond: (j = (a.i + b.i)) [4]
--  Optimizer: Postgres query optimizer
-- (14 rows)
--
-- The crucal parts are:
--
-- * Nested Loop join on the inner side of another Nested Loop join [1], [2]
--
-- * Motion on the outer side of the inner Nested Loop join (the Broadcast
--   Motion on top of "Seq Scan on a" [3])
--
-- * An Index scan in the innermost path, which uses an executor parameter
--   from the outermost path ("b.i", in the Index Cond) [4]
--
-- There must be a Materialize node on top of the "Broadcast Motion -> Seq Scan"
-- path [5]. Otherwise, when the outermost scan on 'b' produces a new row, and
-- the outer Nested Loop calls Rescan on its inner side, the Motion node would
-- be rescanned. Note that the Materialize node at [6] does *not* shield the
-- Motion node from rescanning! That Materialize node is rescanned, when the
-- executor parameter 'b.i' changes.

explain (costs off) select * from a, b, c where b.i = a.i and (a.i + b.i) = c.j;

-- select * from a, b, c where b.i = a.i and (a.i + b.i) = c.j;

create type mytype as (x int, y int);
alter table b add column mt_col mytype;
explain select a.*, b.i, c.* from a, b, c where ((mt_col).x > a.i or b.i = a.i) and (a.i + b.i) = c.j;
select a.*, b.i, c.* from a, b, c where ((mt_col).x > a.i or b.i = a.i) and (a.i + b.i) = c.j;
@kainwen kainwen added the type: bug label Mar 4, 2020
kainwen added a commit to kainwen/gpdb that referenced this issue Mar 4, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit to kainwen/gpdb that referenced this issue Mar 5, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit to kainwen/gpdb that referenced this issue Mar 6, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit to kainwen/gpdb that referenced this issue Mar 6, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit that referenced this issue Mar 6, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue #9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit to kainwen/gpdb that referenced this issue Mar 10, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
kainwen added a commit that referenced this issue Mar 10, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue #9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
@kainwen

This comment has been minimized.

Copy link
Member Author

@kainwen kainwen commented Mar 10, 2020

This issue has been fixed in master (#9680 and #9680)
and 6X (#9709).

5X does not have the issue because 5X does not support lateral and there is no nestParam in Nestloop in 5X.

@kainwen kainwen closed this Mar 10, 2020
weinan003 added a commit to weinan003/gpdb that referenced this issue Mar 15, 2020
To avoid motion deadlock, Greenplum may decide to prefetch
joinqual or inner plan. However, for NestLoop join, inner
plan or joinqual may depend on outerParams. Previously, we
do not handle outerParams correct in prefetch logic and thus
may lead to PANIC. See Github Issue greenplum-db#9679
for Details.

This commit fixes this by faking the outertuple to a null tuple
and then build the params in econtext for NestLoop join's prefetch
logic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.