Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Issue with queries involving small numeric(m,n) fields as keys #2438

Closed
ajcaldera1 opened this issue Sep 26, 2019 · 2 comments
Closed
Assignees
Labels
area/ysql Yugabyte SQL (YSQL) kind/bug This issue is a bug priority/high High Priority
Projects

Comments

@ajcaldera1
Copy link
Contributor

ajcaldera1 commented Sep 26, 2019

UPDATE: Issue is not specific to numeric type. It affects tables if the number of columns in PRIMARY KEY is 3 or more and the order in which columns are specified in CREATE TABLE is different from the order of those columns in the PRIMARY KEY clause.

mydb=> create table c_food (c_id numeric(5,0), c_w_id numeric(4,0), c_d_id numeric(2,0), primary key(c_w_id, c_d_id, c_id));
CREATE TABLE
mydb=> insert into c_food values(9902,89,1);
INSERT 0 1
mydb=> select * from c_food where c_d_id = 1;
ERROR:  numeric field overflow
DETAIL:  A field with precision 2, scale 0 must round to an absolute value less than 10^2.
mydb=> select * from c_food where c_d_id = cast(1 as numeric(2,0));
ERROR:  numeric field overflow
DETAIL:  A field with precision 2, scale 0 must round to an absolute value less than 10^2.
@ajcaldera1 ajcaldera1 added kind/bug This issue is a bug area/ysql Yugabyte SQL (YSQL) priority/high High Priority labels Sep 26, 2019
@ajcaldera1 ajcaldera1 added this to To do in YSQL via automation Sep 26, 2019
@kmuthukk
Copy link
Collaborator

thx for finding/reporting this @ajcaldera1 !

@ndeodhar
Copy link
Contributor

Good find @ajcaldera1 !
This seems to be similar to #1250 which happens if the order in which columns are specified while creating table don't match the order in which primary keys are specified. @d-uspenskiy had fixed the bug during table creation. It looks like we have a similar bug on the INSERT/SELECT path.

If we change the CREATE TABLE column ordering to match PK ordering, then it works:

create table c_food_2 (c_w_id numeric(4,0), c_d_id numeric(2,0), c_id numeric(5,0), primary key(c_w_id, c_d_id, c_id));
insert into c_food_2 values(89,1,9902);
select * from c_food_2 where c_d_id = 1;
 c_w_id | c_d_id | c_id 
--------+--------+------
     89 |      1 | 9902

@ndeodhar ndeodhar assigned d-uspenskiy and unassigned m-iancu and ndeodhar Sep 26, 2019
@ndeodhar ndeodhar moved this from To do to In progress in YSQL Oct 2, 2019
d-uspenskiy added a commit that referenced this issue Oct 5, 2019
Summary:
In case of inserting new value into table `pggate` combines all the fields related to primary key in a single binary value.
`pggate` uses Postgres's native argument order (column order in table creation statement) to build this value.
Combined value is sent to DocDB layer in `ybctid_column_value` field of `PgsqlWriteRequestPB` structure.
DocDB treats value of `ybctid_column_value` field as compound key with respect to DocDB column creation order.
In case DocDB column order differs from Postgres column order primary key field values will be messed up.

Example:
- `CREATE TABLE t(k4 int, k3 int, k2 int, k1 int, value int, PRIMARY KEY(k1, k2, k3, k4))`
Postgres argument order is `k4`, `k3`, `k2`, `k1` (same as column order). DocDB column order is: `k1`, `k2`, `k3`, `k4` (same as compound primary key)

- `INSERT INTO t VALUES(4, 3, 2, 1, 100)`
`pggate` forms `ybctid` value from `k4`=4, `k3`=3, `k2`=2, `k1`=1 sequence. But with one major detail. As `k1` is a first column in compound key it is hash column and its value will goes first. So `pggate` will create `ybctid` value as `<1, 4, 3, 2>`. As DocDB has its own column order from its point of view `<1, 4, 3, 2>` means `k1`=1, `k2`=4, `k3`=3, `k4`=2

- `SELECT * FROM t`
DocDB layer returns `k1`=1, `k2`=4, `k3`=3, `k4`=2 row, but expected is `k1`=1, `k2`=2, `k3`=3, `k4`=4

Current example shows value entanglement but crash is possible in case primary keys fields has different type
```
CREATE TABLE t(k3 int, k2 text, k1 int, PRIMARY KEY(k1, k2, k3))
INSERT INTO t VALUES(3, "test", 1)
```

To solve the issue `pggate` must use DocDB's column order for building `ybctid` value

Test Plan:
New unit test was introduced
```
yb_build.sh --java-test org.yb.pgsql.TestPgRegressMisc
```

Reviewers: mihnea, neil, neha

Reviewed By: neha

Subscribers: alan, kannan, yql

Differential Revision: https://phabricator.dev.yugabyte.com/D7307
YSQL automation moved this from In progress to Done Oct 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ysql Yugabyte SQL (YSQL) kind/bug This issue is a bug priority/high High Priority
Projects
YSQL
  
Done
Development

No branches or pull requests

5 participants