[Bug][Sort] S3DirtySink flushes too quickly

[Yizhou-Yang]

What happened

For jdbc multiple sink and other multiple sinks, when invoke is more frequent then the scheduler's batch interval, the dirty data will flush at a faster rate.

What you expected to happen

let's say the batch interval is 60000. if I call dirtySink.invoke every 1 second, it will invoke, then flush 1-2 records every one second, instead of the 60s which the batch interval is supposed to be.

How to reproduce

let's say the batch interval is 60000. if I call dirtySink.invoke every 1 second, it will invoke, then flush 1-2 records every one second, instead of the 60s which the batch interval is supposed to be.

Environment

No response

InLong version

master

InLong Component

InLong Sort

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct