Too many logs: "synchronize is taking too long, report a bug" occupied a lot disk capacity

[Tammyxia]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.

• Workload: DML + 20 new creating tables.
• TiKV scale-in from 24 to 20. Total region 20w.

2. What did you expect to see?

3. What did you see instead?

• Owner node new generated 230+ cdc log in 20hours, each log file is 300MB, which occupied too much disk capacity.

4. Versions of the cluster

   • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

     4.0.14
     

   • TiCDC version (execute cdc version):

     4.0.14 cloud
     

[liuzix]

This seems to be the exact problem solved by #2078

[overvenus]

@liuzix Can we set the level to debug or reduce frequency? In this case, it takes lots of CPU and IO and may block the entire TiCDC process.

[amyangfei]

Should add this commit to master branch pingcap/ticdc@d6ac981

[overvenus]

@liuzix Please cherry-pick pingcap/ticdc@d6ac981 to master branch, thanks!