[CDCSDK] Alter table support broken with packed row #15063
Assignees
Projects
Comments
sureshdash2022-yb
added a commit
that referenced
this issue
Nov 22, 2022
Summary: During the unit test case failure analysis we observed that for packed row decoding CDC need the corresponding schema and schema version in the //SchemaPackingStorage// object so that in the later stage it can decode the packed row in //PopulatePackedRows// method. but in the method //PopulateCDCSDKWriteRecord//, it is always taking the latest schema version, causing this issue. to handle this we will then pass the corresponding schema version, that CDC is collected from the system catalog table to //PopulateCDCSDKWriteRecord// method Test Plan: Running all the alter table unit testcases with FLAGS_ysql_enable_packed_row as true Reviewers: skumar, srangavajjula, abharadwaj Reviewed By: abharadwaj Subscribers: ycdcxcluster Differential Revision: https://phabricator.dev.yugabyte.com/D21246
sureshdash2022-yb
added a commit
that referenced
this issue
Nov 22, 2022
…DC snapshot mode and for before image type Summary: "Original commit: 3c7146e/D21213 876753e/D21246" During code analysis, we observed that compaction is by default restricted for the active CDC stream using the //cdc_sdk_safe_time//, which really does not require all the features of the CDC and It will unnecessarily consume disk space as well as impact the performance. So in this diff, we will only restrict the compaction:- 1. During snapshot operation. 2. when Before image is enabled for the stream. We have added a few unit test cases around it. [#15063]CDCSDK: Alter table support is broken with packed row During the unit test case failure analysis we observed that for packed row decoding CDC need the corresponding schema and schema version in the //SchemaPackingStorage// object so that in the later stage it can decode the packed row in //PopulatePackedRows// method. but in the method //PopulateCDCSDKWriteRecord//, it is always taking the latest schema version, causing this issue. to handle this we will then pass the corresponding schema version, that CDC is collected from the system catalog table to //PopulateCDCSDKWriteRecord// method Test Plan: ./yb_build.sh --cxx-test integration-tests_cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestCompactionWithSnapshotAndNoBeforeImage ./yb_build.sh --cxx-test integration-tests_cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestExpiredStreamWithCompaction ./yb_build.sh --cxx-test integration-tests_cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestCompactionWithoutBeforeImage Running all the alter table unit testcases with FLAGS_ysql_enable_packed_row as true Reviewers: srangavajjula, skumar, abharadwaj Reviewed By: abharadwaj Subscribers: ycdcxcluster Differential Revision: https://phabricator.dev.yugabyte.com/D21275
|
We have fixed this issue and landed it |
jayant07-yb
pushed a commit
to jayant07-yb/yugabyte-db
that referenced
this issue
Dec 7, 2022
Summary: During the unit test case failure analysis we observed that for packed row decoding CDC need the corresponding schema and schema version in the //SchemaPackingStorage// object so that in the later stage it can decode the packed row in //PopulatePackedRows// method. but in the method //PopulateCDCSDKWriteRecord//, it is always taking the latest schema version, causing this issue. to handle this we will then pass the corresponding schema version, that CDC is collected from the system catalog table to //PopulateCDCSDKWriteRecord// method Test Plan: Running all the alter table unit testcases with FLAGS_ysql_enable_packed_row as true Reviewers: skumar, srangavajjula, abharadwaj Reviewed By: abharadwaj Subscribers: ycdcxcluster Differential Revision: https://phabricator.dev.yugabyte.com/D21246
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: