New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MINOR: Further reduce runtime for metrics integration tests #8514
MINOR: Further reduce runtime for metrics integration tests #8514
Conversation
Checkstyle error:
|
new Properties() | ||
), | ||
STREAM_INPUT_TWO, | ||
Collections.singleton(new KeyValue<>(1, "A")), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we write into STREAM_INPUT_TWO
with 3 calls instead of just one call passing in all 3 records at once?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note we use a mocktime with auto tick WINDOW_SIZE.toMillis()
: and each time its milliseconds()
is called it would auto-advance, we need the produced records with those advanced timestamps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM. Feel free to merge after checkstyle is fixed.
…educe-metrics-integration-runtime
() -> "RocksDB metric bytes.written.total did not increase in " + TIMEOUT + " ms" | ||
); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! I forgot to delete this in my refactoring.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@guozhangwang LGTM
Truly awesome!
Merged to trunk. |
* apache-github/trunk: MINOR: Fix grammar in error message for InvalidRecordException (apache#8465) KAFKA-9868: Reduce number of transaction log partitions for embed broker (apache#8522) MINOR: Adding github whitelist (apache#8523) MINOR: Upgrade gradle plugins and test libraries for Java 14 support (apache#8519) MINOR: Further reduce runtime for metrics integration tests (apache#8514) MINOR: Use .asf.yaml to direct github notifications to JIRA and mailing lists (apache#8521) MINOR: Update to Gradle 6.3 (apache#7677) HOTFIX: fix checkstyle error of RocksDBStoreTest and flaky RocksDBTimestampedStoreTest.shouldOpenExistingStoreInRegularMode (apache#8515) MINOR: cleanup RocksDBStore tests (apache#8510) KAFKA-9818: Fix flaky test in RecordCollectorTest (apache#8507) MINOR: reduce impact of trace logging in replica hot path (apache#8468) KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence (apache#8475) KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test (apache#8501) MINOR: improve test coverage for dynamic LogConfig(s) (apache#7616) MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (apache#8505) KAFKA-9819: Fix flaky test in StoreChangelogReaderTest (apache#8488)
In both RocksDBMetrics and Metrics integration tests, we do not need to wait for consumer to consume records from output topics since the sensors / metrics are registered upon task creation.
Merged the two test cases of RocksDB with one app that creates two state stores (non-segmented and segmented).
With these two changes, local runtime of these two tests reduced from 2min+ and 3min+ to under a minute.
Committer Checklist (excluded from commit message)