Geo-Partitioned tables get created with incorrect tablet counts. #16508
Assignees
Labels
area/ysql
Yugabyte SQL (YSQL)
jira-originated
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Comments
yugabyte-ci
added
area/ysql
emhna
added a commit
that referenced
this issue
Jun 7, 2023
…reated within geo-partitoned tablespace Summary: User tables created within geo-partitioned tablespace use incorrect number of tablet count. Today it uses total number of tablets across the entire cluster to determine the total number of tablets to assign on the table. However, we should use total number of tablets that is assigned on that particular tablespace instead of the entire cluster if the user table is created within a tablespace. The number of tablets is assigned within yb-client side which needs exposure to master side logic such as the TServer placement information in order to fetch the tablets assigned within a particular tablespace. Thus an RPC API is added on the catalog_manager to fetch the proper tablet size and allow the yb-client to use the information when creating a table. Jira: DB-5766 Test Plan: Unit test added. Conducted Dev Portal test with 2, 4, 8 CPUs (both with and without `enable_automatic_tablet_splitting` flag set) to ensure the CPU logic is adhered. Performed table creation with tablespaces in 1, 2, 3 different regions inside each cluster type. Correct number of tablets and tservers were returned. Also created a cluster with different master/tserver CPU sizes and verified CPU size from tserver was chosen over master when calculating number of tablets which is expected. Reviewers: esheng, tverona, myang Reviewed By: myang Subscribers: cdavid, yql, bogdan Differential Revision: https://phorge.dev.yugabyte.com/D24772
emhna
added a commit
that referenced
this issue
Jul 11, 2023
…n user tables created within geo-partitoned tablespace Summary: User tables created within geo-partitioned tablespace use incorrect number of tablet count. Today it uses total number of tablets across the entire cluster to determine the total number of tablets to assign on the table. However, we should use total number of tablets that is assigned on that particular tablespace instead of the entire cluster if the user table is created within a tablespace. The number of tablets is assigned within yb-client side which needs exposure to master side logic such as the TServer placement information in order to fetch the tablets assigned within a particular tablespace. Thus an RPC API is added on the catalog_manager to fetch the proper tablet size and allow the yb-client to use the information when creating a table. Jira: DB-5766 Original commit: `d076617e341162e2d367e6c75443b9cdf0e78978` / `D24772` Test Plan: Unit test added. Conducted Dev Portal test with 2, 4, 8 CPUs (both with and without `enable_automatic_tablet_splitting` flag set) to ensure the CPU logic is adhered. Performed table creation with tablespaces in 1, 2, 3 different regions inside each cluster type. Correct number of tablets and tservers were returned. Also created a cluster with different master/tserver CPU sizes and verified CPU size from tserver was chosen over master when calculating number of tablets which is expected. Reviewers: esheng, tverona, myang Reviewed By: myang Subscribers: bogdan, yql, cdavid Differential Revision: https://phorge.dev.yugabyte.com/D26199
emhna
added a commit
that referenced
this issue
Jul 11, 2023
…n user tables created within geo-partitoned tablespace Summary: User tables created within geo-partitioned tablespace use incorrect number of tablet count. Today it uses total number of tablets across the entire cluster to determine the total number of tablets to assign on the table. However, we should use total number of tablets that is assigned on that particular tablespace instead of the entire cluster if the user table is created within a tablespace. The number of tablets is assigned within yb-client side which needs exposure to master side logic such as the TServer placement information in order to fetch the tablets assigned within a particular tablespace. Thus an RPC API is added on the catalog_manager to fetch the proper tablet size and allow the yb-client to use the information when creating a table. Jira: DB-5766 Original commit: d076617/D24772 Test Plan: Unit test added. Conducted Dev Portal test with 2, 4, 8 CPUs (both with and without `enable_automatic_tablet_splitting` flag set) to ensure the CPU logic is adhered. Performed table creation with tablespaces in 1, 2, 3 different regions inside each cluster type. Correct number of tablets and tservers were returned. Also created a cluster with different master/tserver CPU sizes and verified CPU size from tserver was chosen over master when calculating number of tablets which is expected. Reviewers: esheng, tverona, myang Reviewed By: myang Subscribers: bogdan, yql, cdavid Differential Revision: https://phorge.dev.yugabyte.com/D26310
emhna
added a commit
that referenced
this issue
Jul 12, 2023
…n user tables created within geo-partitoned tablespace Summary: User tables created within geo-partitioned tablespace use incorrect number of tablet count. Today it uses total number of tablets across the entire cluster to determine the total number of tablets to assign on the table. However, we should use total number of tablets that is assigned on that particular tablespace instead of the entire cluster if the user table is created within a tablespace. The number of tablets is assigned within yb-client side which needs exposure to master side logic such as the TServer placement information in order to fetch the tablets assigned within a particular tablespace. Thus an RPC API is added on the catalog_manager to fetch the proper tablet size and allow the yb-client to use the information when creating a table. Jira: DB-5766 Original commit: d076617/D24772 Test Plan: Unit test added. Conducted Dev Portal test with 2, 4, 8 CPUs (both with and without `enable_automatic_tablet_splitting` flag set) to ensure the CPU logic is adhered. Performed table creation with tablespaces in 1, 2, 3 different regions inside each cluster type. Correct number of tablets and tservers were returned. Also created a cluster with different master/tserver CPU sizes and verified CPU size from tserver was chosen over master when calculating number of tablets which is expected. Reviewers: esheng, tverona, myang Reviewed By: myang Subscribers: bogdan, yql, cdavid Differential Revision: https://phorge.dev.yugabyte.com/D26309
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-5766
Tables created in a partitioned tablespace use the "shards per tserver" setting based on total nodes, not the tablespace.
This was confirmed on customer’s prod cluster, where the partitioned sub-table for a primary table was created with 15 tablets, which is the global tserver count for the cluster, not the count of tservers eligible to host the partitioned tablespace
The text was updated successfully, but these errors were encountered: