Fix deltastreamer multi-writer checkpoint mechanism #15926
Description
if we have multiple deltastreamer writers w/ out of order commits, chances that checkpoint management can miss things and can cause dups. we need to fix it properly.
we need to make a fix similar to how we fixed it for spark streaming sink #8558
JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6159
- Type: Bug
- Fix version(s):
- 1.1.0