Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the events are dispatched to different partitions unexpectedly #728

Closed
zier-one opened this issue Jul 8, 2020 · 0 comments
Closed

the events are dispatched to different partitions unexpectedly #728

zier-one opened this issue Jul 8, 2020 · 0 comments
Assignees
Labels
area/ticdc Issues or PRs related to TiCDC. difficulty/medium Medium task. type/bug This is a bug.

Comments

@zier-one
Copy link
Contributor

zier-one commented Jul 8, 2020

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error.

There is a table in upstream TiDB:

CREATE TABLE `mars_task_verify_statistics` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `resource_id` varchar(24) NOT NULL,
  `task_id` bigint(20) NOT NULL,
  `source` varchar(36) NOT NULL,
  `category` varchar(26) DEFAULT NULL,
  `priority` smallint(6) DEFAULT 0,
  `type` smallint(6) NOT NULL,
  `task_time` timestamp NOT NULL,
  `start_time` timestamp NULL DEFAULT NULL,
  `end_time` timestamp NULL DEFAULT NULL,
  `reason` varchar(256) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `idx_resource_id` (`resource_id`),
  UNIQUE KEY `u_source_task` (`source`,`task_id`),
  KEY `idx_task_time` (`task_time`),
  KEY `idx_end_time` (`end_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin

the dispatcher of this table is rowid, we received the following message from Kafka:

[Thread-0][partition-0]
Total Bytes: 159
[CanalEntry]:
header {
  version: 1
  serverenCode: "UTF-8"
  executeTime: 1593749327652
  sourceType: MYSQL
  schemaName: "mars"
  tableName: "mars_task_verify_statistics"
  eventType: DELETE
  props {
    key: "rowsCount"
    value: "1"
  }
}
entryType: ROWDATA
storeValue: "\020\003P\000bB\n$\020\f\032\006source \0010\000B\024user_info_after_pass\n\032\020\003\032\atask_id \0010\000B\t229209952"

[RowChange]:
eventType: DELETE
isDdl: false
rowDatas {
  beforeColumns {
    sqlType: 12
    name: "source"
    isKey: true
    isNull: false
    value: "user_info_after_pass"
  }
  beforeColumns {
    sqlType: 3
    name: "task_id"
    isKey: true
    isNull: false
    value: "229209952"
  }
}

[Thread-0][partition-28]
Total Bytes: 113
[CanalEntry]:
header {
  version: 1
  serverenCode: "UTF-8"
  executeTime: 1593749327652
  sourceType: MYSQL
  schemaName: "mars"
  tableName: "mars_task_verify_statistics"
  eventType: DELETE
  props {
    key: "rowsCount"
    value: "1"
  }
}
entryType: ROWDATA
storeValue: "\020\003P\000b\026\n\024\020\003\032\002id \0010\000B\b22647228"

[RowChange]:
eventType: DELETE
isDdl: false
rowDatas {
  beforeColumns {
    sqlType: 3
    name: "id"
    isKey: true
    isNull: false
    value: "22647228"
  }
}

we can found that the delete events of one row are dispatched to different partitions, it's not expected

  1. Versions of the cluster

v4.0.2

@zier-one zier-one added the type/bug This is a bug. label Jul 8, 2020
@zier-one zier-one self-assigned this Jul 8, 2020
@amyangfei amyangfei added the difficulty/medium Medium task. label Jul 29, 2020
@AkiraXie AkiraXie added the area/ticdc Issues or PRs related to TiCDC. label Mar 9, 2022
@github-actions github-actions bot added this to Need Triage in Question and Bug Reports Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ticdc Issues or PRs related to TiCDC. difficulty/medium Medium task. type/bug This is a bug.
Projects
Development

No branches or pull requests

3 participants