Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-8613: KIP-633 New APIs for Controlling Grace Period for Windowed Operations #10924

Conversation

izzyacademy
Copy link
Contributor

KIP-633 New APIs for Controlling Grace Period for Windowed Operations

  • Added API changes by KIP-633 for JoinWindows, SessionWindows, TimeWindows and SlidingWindows
  • Renamed Windows.DEFAULT_GRACE_PERIOD_MS to DEPRECATED_OLD_24_HR_GRACE_PERIOD
  • Added new constant Windows.NO_GRACE_PERIOD to avoid magic constants when 0 is specified as grace Period
  • Added preliminary Java unit test cases for new API methods
  • Replaced Deprecated calls with equivalent in Examples
  • Replaced Deprecated API calls in Scala tests with updated API method calls
  • Added Deprecation suppression in Tests for derecated API method calls in Java and Scala Tests

modified: streams/src/main/java/org/apache/kafka/streams/kstream/JoinWindows.java
modified: streams/src/main/java/org/apache/kafka/streams/kstream/SessionWindows.java
modified: streams/src/main/java/org/apache/kafka/streams/kstream/SlidingWindows.java
modified: streams/src/main/java/org/apache/kafka/streams/kstream/TimeWindows.java
modified: streams/src/main/java/org/apache/kafka/streams/kstream/Windows.java

modified: streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java
modified: streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewUntypedDemo.java
modified: streams/examples/src/main/java/org/apache/kafka/streams/examples/temperature/TemperatureDemo.java

modified: streams/streams-scala/src/main/scala/org/apache/kafka/streams/scala/kstream/KStream.scala
modified: streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/TopologyTest.scala
modified: streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala
modified: streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KTableTest.scala

modified: streams/src/test/java/org/apache/kafka/streams/StreamsBuilderTest.java
modified: streams/src/test/java/org/apache/kafka/streams/TopologyTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/AbstractResetIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/InternalTopicIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/JoinStoreIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/KStreamAggregationDedupIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/KStreamAggregationIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/KStreamRepartitionIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/MetricsIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java
modified: streams/src/test/java/org/apache/kafka/streams/integration/RocksDBMetricsIntegrationTest.java

modified: streams/src/test/java/org/apache/kafka/streams/kstream/JoinWindowsTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/RepartitionTopicNamingTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/SessionWindowsTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/SlidingWindowsTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/TimeWindowsTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KGroupedStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamLeftJoinTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamOuterJoinTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamRepartitionTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregateTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregateTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SessionWindowedCogroupedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SessionWindowedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SlidingWindowedCogroupedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SlidingWindowedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SuppressScenarioTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/SuppressTopologyTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowedCogroupedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowedKStreamImplTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/graph/GraphGraceSearchUtilTest.java
modified: streams/src/test/java/org/apache/kafka/streams/kstream/internals/graph/StreamsGraphTest.java
modified: streams/src/test/java/org/apache/kafka/streams/processor/internals/RepartitionOptimizingTest.java
modified: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
modified: streams/src/test/java/org/apache/kafka/streams/tests/SmokeTestClient.java
modified: streams/src/test/java/org/apache/kafka/streams/tests/StreamsOptimizedTest.java
modified: streams/src/test/java/org/apache/kafka/test/GenericInMemoryKeyValueStore.java
modified: streams/src/test/java/org/apache/kafka/test/GenericInMemoryTimestampedKeyValueStore.java

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.

Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

KIP-633 New APIs for Controlling Grace Period for Windowed Operations

- Added API changes by KIP-633 for JoinWindows, SessionWindows, TimeWindows and SlidingWindows
- Renamed Windows.DEFAULT_GRACE_PERIOD_MS to DEPRECATED_OLD_24_HR_GRACE_PERIOD
- Added new constant Windows.NO_GRACE_PERIOD to avoid magic constants when 0 is specified as grace Period
- Added preliminary Java unit test cases for new API methods
- Replaced Deprecated calls with equivalent in Examples
- Replaced Deprecated API calls in Scala tests with updated API method calls
- Added Deprecation suppression in Tests for derecated API method calls in Java and Scala Tests

modified:   streams/src/main/java/org/apache/kafka/streams/kstream/JoinWindows.java
modified:   streams/src/main/java/org/apache/kafka/streams/kstream/SessionWindows.java
modified:   streams/src/main/java/org/apache/kafka/streams/kstream/SlidingWindows.java
modified:   streams/src/main/java/org/apache/kafka/streams/kstream/TimeWindows.java
modified:   streams/src/main/java/org/apache/kafka/streams/kstream/Windows.java

modified:   streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java
modified:   streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewUntypedDemo.java
modified:   streams/examples/src/main/java/org/apache/kafka/streams/examples/temperature/TemperatureDemo.java

modified:   streams/streams-scala/src/main/scala/org/apache/kafka/streams/scala/kstream/KStream.scala
modified:   streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/TopologyTest.scala
modified:   streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala
modified:   streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KTableTest.scala

modified:   streams/src/test/java/org/apache/kafka/streams/StreamsBuilderTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/TopologyTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/AbstractResetIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/InternalTopicIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/JoinStoreIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/KStreamAggregationDedupIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/KStreamAggregationIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/KStreamRepartitionIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/MetricsIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/integration/RocksDBMetricsIntegrationTest.java

modified:   streams/src/test/java/org/apache/kafka/streams/kstream/JoinWindowsTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/RepartitionTopicNamingTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/SessionWindowsTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/SlidingWindowsTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/TimeWindowsTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KGroupedStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamLeftJoinTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamOuterJoinTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamRepartitionTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregateTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregateTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SessionWindowedCogroupedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SessionWindowedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SlidingWindowedCogroupedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SlidingWindowedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SuppressScenarioTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/SuppressTopologyTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowedCogroupedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/TimeWindowedKStreamImplTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/graph/GraphGraceSearchUtilTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/kstream/internals/graph/StreamsGraphTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/processor/internals/RepartitionOptimizingTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
modified:   streams/src/test/java/org/apache/kafka/streams/tests/SmokeTestClient.java
modified:   streams/src/test/java/org/apache/kafka/streams/tests/StreamsOptimizedTest.java
modified:   streams/src/test/java/org/apache/kafka/test/GenericInMemoryKeyValueStore.java
modified:   streams/src/test/java/org/apache/kafka/test/GenericInMemoryTimestampedKeyValueStore.java
@izzyacademy
Copy link
Contributor Author

@ableegoldman @mjsax @showuon @cadonna

When you have a moment, please take a look.

This is an update based on the feedback from PR #10740

@izzyacademy
Copy link
Contributor Author

I will close this one and resolve the conflicts before opening the next one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants