[CDCSDK] [PG Parity] Consistency broke when run with nemesis: Received restart LSN 1 is less than the last received restart LSN 185267

[shamanthchandra-yb]

Jira Link: DB-10866

Description

Please refer JIRA for stress run link.

AssertionError: Some error, check test log. Reasons: ["Total sum is not as expected, consistency broke! Expected=1000000, Actual=998011. Thread: 0"]

Connector log ERROR:

2024-04-18 12:00:14,997 ERROR  Postgres|db_cdc|streaming  Producer failure   [io.debezium.pipeline.ErrorHandler]
com.yugabyte.util.PSQLException: ERROR: UpdateAndPersistLSN failed for stream_id: a191317acc156086df4e3c78531e1487: Received restart LSN 1 is less than the last received restart LSN 185267
	at com.yugabyte.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
	at com.yugabyte.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1263)
	at com.yugabyte.core.v3.QueryExecutorImpl.readFromCopy(QueryExecutorImpl.java:1163)
	at com.yugabyte.core.v3.CopyDualImpl.readFromCopy(CopyDualImpl.java:44)
	at com.yugabyte.core.v3.replication.V3PGReplicationStream.receiveNextData(V3PGReplicationStream.java:160)
	at com.yugabyte.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:125)
	at com.yugabyte.core.v3.replication.V3PGReplicationStream.readPending(V3PGReplicationStream.java:82)
	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:622)
	at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.processMessages(PostgresStreamingChangeEventSource.java:221)
	at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:183)
	at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:37)
	at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:271)
	at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:194)
	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:137)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Source connector version

fourpointfour/ybdb-debezium:0.3

Connector configuration

add connector connector_name='yugabyte_connector' stream_id='8ca0f8fd4bd4a48b82496f61ad154960' db_name='cdc_648b7a' connector_host='172.151.26.0' table_list=['test_cdc_f61ed2_0', 'test_cdc_f61ed2_1', 'test_cdc_f61ed2_2', 'test_cdc_f61ed2_3', 'test_cdc_f61ed2_4'] {'name': 'yugabyte_connector', 'config': {'database.master.addresses': '172.151.29.120:7100,172.151.23.105:7100,172.151.31.178:7100', 'database.hostname': '172.151.29.120:5433,172.151.23.105:5433,172.151.31.178:5433', 'database.port': 5433, 'database.masterhost': '172.151.23.105', 'database.masterport': '7100', 'database.user': 'yugabyte', 'database.password': 'yugabyte', 'database.dbname': 'cdc_648b7a', 'snapshot.mode': 'never', 'admin.operation.timeout.ms': 600000, 'socket.read.timeout.ms': 300000, 'max.connector.retries': '10', 'operation.timeout.ms': 600000, 'topic.creation.default.compression.type': 'lz4', 'topic.creation.default.cleanup.policy': 'delete', 'topic.creation.default.partitions': 1, 'topic.creation.default.replication.factor': '1', 'tasks.max': '1', 'connector.class': 'io.debezium.connector.postgresql.PostgresConnector', 'topic.prefix': 'db_cdc', 'plugin.name': 'pgoutput', 'slot.name': '8ca0f8fd4bd4a48b82496f61ad154960_from_con', 'publication.autocreate.mode': 'filtered', 'publication.name': 'pn_yugabyte_connector', 'table.include.list': 'public.test_cdc_f61ed2_0,public.test_cdc_f61ed2_1,public.test_cdc_f61ed2_2,public.test_cdc_f61ed2_3,public.test_cdc_f61ed2_4', 'transforms': 'Reroute', 'transforms.Reroute.topic.regex': '(.*)', 'transforms.Reroute.topic.replacement': 'db_cdc_all_events', 'transforms.Reroute.type': 'io.debezium.transforms.ByLogicalTableRouter', 'transforms.Reroute.key.field.regex': 'db_cdc(.*)', 'transforms.Reroute.key.field.replacement': '$1', 'max.poll.interval.ms': '5000', 'transaction.ordering': 'true', 'provide.transaction.metadata': 'true'}}

YugabyteDB version

2.23.0.0-b131

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.