[SPARK-30311][SS] Use KafkaConsumer.poll(Duration) instead of deprecated one#26954
[SPARK-30311][SS] Use KafkaConsumer.poll(Duration) instead of deprecated one#26954dongjoon-hyun wants to merge 2 commits intoapache:masterfrom dongjoon-hyun:SPARK-30311
Conversation
| package org.apache.spark.sql.kafka010 | ||
|
|
||
| import java.{util => ju} | ||
| import java.time.{Duration => jDuration} |
There was a problem hiding this comment.
This will avoid conflicts with import scala.concurrent.duration.Duration at line 27.
|
Sorry, but we tried it out and that cannot be simple replacement. Please refer https://issues.apache.org/jira/browse/SPARK-28367 and #25135 We were waiting for Kafka 2.4.0 to deal with |
|
Oh, I didn't notice that effort. Let me take a look. Thanks, @HeartSaVioR . |
|
Test FAILed. |
What changes were proposed in this pull request?
This PR replaces the deprecated
KafkaConsumer.poll(long)withKafkaConsumer.poll(Duration). The old one is deprecated at Kafka 2.0.0.Why are the changes needed?
This will use the latest API.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass the Jenkins.