Skip to content

Commit

Permalink
Handle exceptions while processing files better
Browse files Browse the repository at this point in the history
Now only commit offsets for a file if the entire file is processed without errors, this ensures we process every record and do not miss records
  • Loading branch information
grantatspothero committed May 26, 2020
1 parent a1705c6 commit 2751510
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ public List<SourceRecord> poll() {
} catch (ConnectException | IOException e) {
//when an exception happens reading a file, the connector continues
log.error("Error reading file [{}]. Keep going...", metadata.getPath(), e);
return new ArrayList<SourceRecord>().stream();
}
log.debug("Read [{}] records from file [{}].", records.size(), metadata.getPath());

Expand Down

0 comments on commit 2751510

Please sign in to comment.