How to get metrics/stats of completed and/or running transactions #199

Open
muga opened this Issue Jun 12, 2015 · 0 comments

Projects

None yet

1 participant

@muga
Contributor
muga commented Jun 12, 2015

Current Embulk doesn't provide metrics/states of the done BulkLoader when it was finished. For example, almost users want to know how many number of records successfully loaded (or skipped) by Embulk.

To do that, I think that we can use CommitReport. When each input/output plugin finished, it add the information of the number of the loaded (or skipped) records to CommitReport. For example, output plugin's CommitReport is like {"loaded_records": 1000, "skipped_records":1}. Since LoaderState already has the list of CommitReports, it can sum and add the numbers to ExecutionResult.

@muga muga added this to the v0.9 milestone Apr 12, 2016
@muga muga referenced this issue in civitaspo/embulk-filter-expand_json Apr 26, 2016
Closed

Support stop_on_invalid_record option #14

@frsyuki frsyuki changed the title from How to get metrics/stats of the done BulkLoader to How to get metrics/stats of completed and/or running transactions Apr 29, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment