Skip to content

schemastore: GC safepoint mismatch blocks changefeed creation on TiDB Cloud Essential #5785

Description

@pingyu

What did you do?

Run TiCDC new architecture in a TiDB Cloud Essential environment and create/start a changefeed for a keyspace using GC safepoint v2.

Relevant environment and keyspace configuration:

TiCDC: v26.3.3
Environment: TiDB Cloud Essential
enable-legacy-safepoint: true
keyspace safe_point_version: v2
keyspace gc_life_time: 6000
keyspace gc_management_type: absent

During changefeed creation, TiCDC creates an API V2 TiKV store for the keyspace and initializes SchemaStore from the keyspace-v2 minimum service safepoint. PD-CSE does not implement the newer GetGCState RPC, so current client-go falls back to direct etcd reading. Because gc_management_type=keyspace_level_gc is absent, the compatibility loader selects /tidb/store/gcworker/saved_safe_point.

The following sequence reproduces the failure:

  1. GetGCState returns gRPC Unimplemented.
  2. TiCDC obtains keyspace-v2 safepoint 467969442615394304 (2026-07-27 14:15:53.141 UTC).
  3. SchemaStore uses exactly that value as snapTs.
  4. client-go validates the snapshot against transaction safepoint 467969667508994048 (2026-07-27 14:30:11.042 UTC).
  5. meta.ListDatabases() returns error 9006 because snapTs < txnSafePoint.

Cluster, tenant, project, keyspace name, and endpoint identifiers have been omitted from this public report.

What did you expect to see?

SchemaStore and the changefeed should initialize successfully using a transaction safepoint consistent with the keyspace-v2 GC state.

If the snapshot is irrecoverably older than the transaction safepoint, TiCDC should terminate the bootstrap attempt promptly with an actionable error instead of retrying the same timestamp forever.

What did you see instead?

TiCDC continuously reports this warning at approximately 100 ms intervals:

[WARN] [disk_format.go:599] ["list databases failed, retrying"] [error="[tikv:9006]GC life time is shorter than transaction duration, transaction start ts is 467969442615394304 (2026-07-27 14:15:53.141 +0000 UTC), txn safe point is 467969667508994048 (2026-07-27 14:30:11.042 +0000 UTC)"]

The HTTP/TCP/gRPC server and coordinator are running, but SchemaStore initialization for this keyspace does not complete, so the corresponding changefeed create/start path remains blocked.

Technical observations:

  • persistSchemaSnapshot creates a new snapshot reader on each attempt but always uses the same fixed snapTs.
  • The ListDatabases error branch sleeps for 100 ms and retries without a bound or context cancellation.
  • The existing isGCLifeTimeError fast-fail check is only applied to the later table metadata read path.
  • PD-CSE supports keyspace-v2 GC/service safepoint APIs but not GetGCState, causing client-go to enter its direct-etcd compatibility path.
  • TiCDC reads a keyspace-v2 minimum service safepoint while client-go visibility validation uses a newer transaction safepoint from the compatibility loader.

Related work:

Suggested actions:

  1. Align the TiCDC, TiDB/client-go, and PD-CSE GC-state API compatibility set so transaction visibility and SchemaStore snapshot selection use consistent keyspace-scoped state.
  2. Backport schemastore: retry with fresh snapshot after GC #5493 to the affected next-gen release branch.
  3. Add an integration test covering safe_point_version=v2, missing GetGCState, and the direct-etcd fallback.

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

Managed TiDB Cloud Essential version; exact TiDB build is not exposed in the available incident data.

Upstream TiKV version (execute tikv-server --version):

Managed TiDB Cloud Essential version; exact TiKV build is not exposed in the available incident data.

TiCDC version (execute cdc version):

Release Version: v26.3.3
Git Commit Hash: 212514835a426c4fed697c6a8d7526799d3ad427
Kernel Type: Next Generation

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions