Measuring throughput of connectors #2605
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cgardens
approved these changes
Mar 24, 2021
airbyte-integrations/connectors/source-appsflyer-singer/unit_tests/unit_test.py
Outdated
Show resolved
Hide resolved
| if (lastAttempt.getOutput() != null && lastAttempt.getOutput().isPresent()) { | ||
| final JobOutput jobOutput = lastAttempt.getOutput().get(); | ||
| final StandardSyncSummary syncSummary = jobOutput.getSync().getStandardSyncSummary(); | ||
| metadata.put("duration", syncSummary.getEndTime() - syncSummary.getStartTime()); |
There was a problem hiding this comment.
just want to sanity check that sync summary metrics are in seconds?
There was a problem hiding this comment.
it's in milliseconds based on what's being saved in the syncSummary:
There was a problem hiding this comment.
if you look at the excel spreadsheet that john put together for the tracking plan he is expecting seconds. either his expectation needs to change to ms or we need to convert here. i have no preference, just want to make sure that the two of you are speaking the same language.
There was a problem hiding this comment.
Ok will switch to seconds, thanks!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Closes #2524
How
Add attempts metadata when tracking job completions
Recommended reading order
airbyte-scheduler/src/main/java/io/airbyte/scheduler/JobTracker.java