Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-29613][BUILD][SS] Upgrade to Kafka 2.3.1 #26271

Closed
wants to merge 1 commit into from

Conversation

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Oct 27, 2019

What changes were proposed in this pull request?

This PR aims to upgrade to Kafka 2.3.1 client library for client fixes like KAFKA-8950, KAFKA-8570, and KAFKA-8635. The following is the full release note.

Why are the changes needed?

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Jenkins with the existing tests.

@SparkQA

This comment has been minimized.

Copy link

SparkQA commented Oct 27, 2019

Test build #112724 has finished for PR 26271 at commit 1ce53bb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@dongjoon-hyun

This comment has been minimized.

Copy link
Member Author

dongjoon-hyun commented Oct 27, 2019

The UT failures are irrelevant to this PR.

@dongjoon-hyun

This comment has been minimized.

Copy link
Member Author

dongjoon-hyun commented Oct 27, 2019

Retest this please.

1 similar comment
@dongjoon-hyun

This comment has been minimized.

Copy link
Member Author

dongjoon-hyun commented Oct 27, 2019

Retest this please.

@SparkQA

This comment has been minimized.

Copy link

SparkQA commented Oct 27, 2019

Test build #112735 has finished for PR 26271 at commit 1ce53bb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
Copy link
Member

srowen left a comment

Looks OK pending tests

@SparkQA

This comment has been minimized.

Copy link

SparkQA commented Oct 27, 2019

Test build #112736 has finished for PR 26271 at commit 1ce53bb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@dongjoon-hyun

This comment has been minimized.

Copy link
Member Author

dongjoon-hyun commented Oct 27, 2019

Thank you, @srowen .
Merged to master.

@dongjoon-hyun dongjoon-hyun deleted the dongjoon-hyun:SPARK-29613 branch Oct 27, 2019
ulysses-you pushed a commit to ulysses-you/spark that referenced this pull request Oct 31, 2019
### What changes were proposed in this pull request?

This PR aims to upgrade to Kafka 2.3.1 client library for client fixes like KAFKA-8950, KAFKA-8570, and KAFKA-8635. The following is the full release note.
- https://archive.apache.org/dist/kafka/2.3.1/RELEASE_NOTES.html

### Why are the changes needed?

- [KAFKA-8950 KafkaConsumer stops fetching](https://issues.apache.org/jira/browse/KAFKA-8950)
- [KAFKA-8570 Downconversion could fail when log contains out of order message formats](https://issues.apache.org/jira/browse/KAFKA-8570)
- [KAFKA-8635 Unnecessary wait when looking up coordinator before transactional request](https://issues.apache.org/jira/browse/KAFKA-8635)

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

Pass the Jenkins with the existing tests.

Closes apache#26271 from dongjoon-hyun/SPARK-29613.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
ulysses-you pushed a commit to ulysses-you/spark that referenced this pull request Oct 31, 2019
### What changes were proposed in this pull request?

This PR aims to upgrade to Kafka 2.3.1 client library for client fixes like KAFKA-8950, KAFKA-8570, and KAFKA-8635. The following is the full release note.
- https://archive.apache.org/dist/kafka/2.3.1/RELEASE_NOTES.html

### Why are the changes needed?

- [KAFKA-8950 KafkaConsumer stops fetching](https://issues.apache.org/jira/browse/KAFKA-8950)
- [KAFKA-8570 Downconversion could fail when log contains out of order message formats](https://issues.apache.org/jira/browse/KAFKA-8570)
- [KAFKA-8635 Unnecessary wait when looking up coordinator before transactional request](https://issues.apache.org/jira/browse/KAFKA-8635)

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

Pass the Jenkins with the existing tests.

Closes apache#26271 from dongjoon-hyun/SPARK-29613.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.