logSink.units does not flush regularly

[dengqee]

Bug Report

look 1259

https://github.com/pingcap/ticdc/blob/04e028419387871b80ddc95377751092f03f26ae/cdc/sink/cdclog/utils.go#L124-L139
There is a ticker to flush logSink.units every 500 ms, but why 1259 did not flush logSink.units?
So, I add a log, like this

I created a s3 sink changefeed:

cdc cli changefeed create --sink-uri="s3://tidb-test/cdclog/?endpoint=http://xxxx:9000&access-Key=xxxx&secret-Access-Key=xxxx"  --changefeed-id=s3-sink --start-ts 426674072187043841 --pd http://xxxx:2379

insert data by sysbench

sysbench --mysql-host=xxxx --mysql-user=root --mysql-port=4000 --mysql-db=test oltp_insert --tables=1 --mysql-password='xxxx' --table-size=200000 prepare

I got this:

https://github.com/pingcap/ticdc/blob/04e028419387871b80ddc95377751092f03f26ae/cdc/sink/cdclog/s3.go#L80-L82
unit.size < 5MB, so shouldFlush return false, but utils.dataCh chennal is full, so changefeed is bloked.

Please answer these questions before submitting your issue. Thanks!

4. Versions of the cluster

   • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

      v4.0.10
     

   • TiCDC version (execute cdc version):

     v4.0.10
     

[amyangfei]

closed by #2431. Thanks for your contribution