Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-14343: Upgrade tests for state updater #12801

Merged
merged 3 commits into from Dec 20, 2022

Conversation

lucasbru
Copy link
Member

@lucasbru lucasbru commented Oct 31, 2022

KAFKA-14343: Upgrade tests for state updater
A test that verifies the upgrade from a version of Streams with
state updater disabled to a version with state updater enabled
and vice versa, so that we can offer a save upgrade path.

  • upgrade test from a version of Streams with state updater
    disabled to a version with state updater enabled
  • downgrade test from a version of Streams with state updater
    enabled to a version with state updater disabled

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@lucasbru lucasbru changed the title KAFKA-14309: FK join upgrades not tested with DEV_VERSION KAFKA-14343: Upgrade tests for state updater Oct 31, 2022
A test that verifies the upgrade from a version of Streams with
state updater disabled to a version with state updater enabled
and vice versa, so that we can offer a save upgrade path.

 - upgrade test from a version of Streams with state updater
disabled to a version with state updater enabled
 - downgrade test from a version of Streams with state updater
 enabled to a version with state updater disabled
@lucasbru
Copy link
Member Author

CI test failures are unrelated.

Passing system test run 10x for the new upgrade / downgrade test:

https://jenkins.confluent.io/job/system-test-kafka-branch-builder/5206/

@cadonna could you have a look?

Copy link
Contributor

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasbru Thanks for the PR!

Here my feedback!

tests/kafkatest/tests/streams/streams_upgrade_test.py Outdated Show resolved Hide resolved
# rolling bounce
random.shuffle(self.processors)
for p in self.processors:
p.CLEAN_NODE_ENABLED = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about cleaning the state on the node before we bounce so that some restoration is going on?
I think setting p.CLEAN_NODE_ENABLED to True would achieve this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea

lucasbru and others added 2 commits December 19, 2022 20:29
Co-authored-by: Bruno Cadonna <cadonna@apache.org>
Copy link
Contributor

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, @lucasbru!

I verified that the system tests pass.

LGTM!

@cadonna
Copy link
Contributor

cadonna commented Dec 20, 2022

Build failures are unrelated:

Build / JDK 17 and Scala 2.13 / kafka.api.PlaintextEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe(String).quorum=zk
Build / JDK 17 and Scala 2.13 / org.apache.kafka.streams.integration.SmokeTestDriverIntegrationTest.[1] true
Build / JDK 17 and Scala 2.13 / org.apache.kafka.streams.integration.SmokeTestDriverIntegrationTest.[1] true
Build / JDK 11 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testEarlyControllerResults()

@cadonna cadonna merged commit c8675d4 into apache:trunk Dec 20, 2022
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
A test that verifies the upgrade from a version of Streams with
state updater disabled to a version with state updater enabled
and vice versa, so that we can offer a save upgrade path.

 - upgrade test from a version of Streams with state updater
disabled to a version with state updater enabled
 - downgrade test from a version of Streams with state updater
 enabled to a version with state updater disabled

Reviewer: Bruno Cadonna <cadonna@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants