KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to CURRENT_BRANCH to reduce confusion. #2457

ewencp · 2017-01-27T03:25:53Z

No description provided.

… CURRENT_BRANCH to reduce confusion.

ewencp · 2017-01-27T03:29:32Z

@ijuma Perhaps you can take a gander at this one. The bulk of the changes are just the renaming we discussed in the thread. I went with a slightly different name because the directory for the code and the name in the ducktape code are semi- tied together. DEV_BRANCH seemed a reasonable compromise.

Buried in the changes are the few test additions. I'm actually not that familiar with those tests and they include parameters that I was a bit unclear on why we have the set of parameterizations we have, so look carefully at those changes.

I kicked off a test for the core system tests and it looks mostly ok here: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/691/consoleFull There are some failures, but some are known (the issue with ProduceConsumeValidate). The others I'm hoping someone can explain -- I'm not sure why a cluster ID in ZK (or lack thereof) would suddenly show up in these tests.

asfbot · 2017-01-27T05:05:36Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/1283/
Test PASSed (JDK 8 and Scala 2.11).

asfbot · 2017-01-27T05:09:28Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/1281/
Test PASSed (JDK 8 and Scala 2.12).

asfbot · 2017-01-27T05:38:19Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/1281/
Test PASSed (JDK 7 and Scala 2.10).

ijuma

Thanks for the PR. Looks good overall, left a few mostly minor comments and an explanation of the cluster id failure.

It may be worth updating the PR title to say DEV_BRANCH instead of CURRENT_BRANCH.

ijuma · 2017-01-27T11:18:03Z

tests/kafkatest/sanity_checks/test_verifiable_producer.py

-        # that this check works with TRUNK
-        # When running VerifiableProducer 0.8.X, both trunk version and 0.8.X should show up because of the way
-        # verifiable producer pulls in some trunk directories into its classpath
+        # that this check works with CURRENT_BRANCH


Should this be DEV_BRANCH?

ijuma · 2017-01-27T11:25:57Z

tests/kafkatest/version.py

        else:
            return LooseVersion.__str__(self)


 def get_version(node=None):
    """Return the version attached to the given node.
-    Default to trunk if node or node.version is undefined (aka None)
+    Default to CURRENT_BRANCH if node or node.version is undefined (aka None)


DEV_BRANCH?

ijuma · 2017-01-27T11:26:14Z

tests/unit/directory_layout/check_project_paths.py

@@ -74,16 +74,16 @@ def check_node_or_version_helper(self):
        """
        resolver = create_path_resolver(DummyContext())

-        # Node with no version attribute should resolve to TRUNK
+        # Node with no version attribute should resolve to CURRENT_BRANCH


DEV_BRANCH

ijuma · 2017-01-27T11:31:42Z

tests/kafkatest/tests/core/upgrade_test.py

@@ -62,6 +62,7 @@ def perform_upgrade(self, from_kafka_version, to_message_format_version=None):
            self.kafka.start_node(node)

    @cluster(num_nodes=6)
+    @parametrize(from_kafka_version=str(LATEST_0_10_1), to_message_format_version=None, compression_types=["lz4"])


Maybe we should have a line for the old consumer as well.

Regarding your question about the cluster id, the following line needs to be conditional on from_kafka_version <= LATEST_0_10_0:

assert self.zk.query("/cluster/id") is None

The code is just checking that the cluster id is generated after an upgrade from 0.10.0.x or lower to 0.10.1.x or higher.

Ack, thanks for the explanation.

ijuma · 2017-01-27T11:33:58Z

tests/kafkatest/tests/core/compatibility_test_new_broker_test.py

+    @parametrize(producer_version=str(LATEST_0_9), consumer_version=str(DEV_BRANCH), compression_types=["snappy"], timestamp_type=None)
+    @parametrize(producer_version=str(DEV_BRANCH), consumer_version=str(LATEST_0_9), compression_types=["snappy"], timestamp_type=str("CreateTime"))
+    @parametrize(producer_version=str(DEV_BRANCH), consumer_version=str(DEV_BRANCH), compression_types=["snappy"], timestamp_type=str("LogAppendTime"))
+    @parametrize(producer_version=str(LATEST_0_10_1), consumer_version=str(LATEST_0_10_1), compression_types=["snappy"],timestamp_type=str("LogAppendTime"))


Nit: space missing before timestamp_type.

ewencp · 2017-01-27T21:50:46Z

Addressed those issues and kicked of one final run to validate it: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/692/

ijuma

Thanks, LGTM (assuming the tests pass).

asfbot · 2017-01-27T22:39:03Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/1304/
Test PASSed (JDK 8 and Scala 2.12).

asfbot · 2017-01-27T23:35:47Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/1306/
Test FAILed (JDK 8 and Scala 2.11).

asfbot · 2017-01-27T23:38:46Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-jdk7-scala2.10/1304/
Test PASSed (JDK 7 and Scala 2.10).

ijuma · 2017-01-28T01:38:48Z

System tests passed, merging to trunk and 0.10.2 branches.

…CH to reduce confusion Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2457 from ewencp/kafka-4450-upgrade-tests

…CH to reduce confusion Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes apache#2457 from ewencp/kafka-4450-upgrade-tests

KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to…

b0d74ef

… CURRENT_BRANCH to reduce confusion.

ijuma reviewed Jan 27, 2017

View reviewed changes

Address review comments.

85aff99

ijuma approved these changes Jan 27, 2017

View reviewed changes

asfgit closed this in 6264cc1 Jan 28, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to CURRENT_BRANCH to reduce confusion. #2457

KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to CURRENT_BRANCH to reduce confusion. #2457

ewencp commented Jan 27, 2017

ewencp commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

ijuma left a comment

ijuma Jan 27, 2017

ijuma Jan 27, 2017

ijuma Jan 27, 2017

ijuma Jan 27, 2017 •

edited

ewencp Jan 27, 2017

ijuma Jan 27, 2017

ewencp commented Jan 27, 2017

ijuma left a comment

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

ijuma commented Jan 28, 2017

KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to CURRENT_BRANCH to reduce confusion. #2457

KAFKA-4450: Add upgrade tests for 0.10.1 releases and rename TRUNK to CURRENT_BRANCH to reduce confusion. #2457

Conversation

ewencp commented Jan 27, 2017

ewencp commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

ijuma left a comment

Choose a reason for hiding this comment

ijuma Jan 27, 2017

Choose a reason for hiding this comment

ijuma Jan 27, 2017

Choose a reason for hiding this comment

ijuma Jan 27, 2017

Choose a reason for hiding this comment

ijuma Jan 27, 2017 • edited

Choose a reason for hiding this comment

ewencp Jan 27, 2017

Choose a reason for hiding this comment

ijuma Jan 27, 2017

Choose a reason for hiding this comment

ewencp commented Jan 27, 2017

ijuma left a comment

Choose a reason for hiding this comment

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

asfbot commented Jan 27, 2017

ijuma commented Jan 28, 2017

ijuma Jan 27, 2017 •

edited