Skip to content

Conversation

@ScrapCodes
Copy link
Member

@ScrapCodes ScrapCodes commented Mar 18, 2020

What changes were proposed in this pull request?

This is a follow up for SPARK-30715 . Kubernetes client version in sync in integration-tests and kubernetes/core

Why are the changes needed?

More than once, the kubernetes client version has gone out of sync between integration tests and kubernetes/core. So brought them up in sync again and added a comment to save us from future need of this additional followup.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Manually.

@SparkQA
Copy link

SparkQA commented Mar 18, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/24711/

@SparkQA
Copy link

SparkQA commented Mar 18, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/24711/

@SparkQA
Copy link

SparkQA commented Mar 18, 2020

Test build #119990 has finished for PR 27948 at commit 0a6b227.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

protected var appLocator: String = _

// Default memory limit is 1024M + 384M (minimum overhead constant)
private val baseMemory = s"${1024 + 384}Mi"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @ScrapCodes . Do we need this change in this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Dongjoon, Yes, it is part of updating the client version. Tests won't pass otherwise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be precise, in 4.7.1 Quantity constructor tries to parse the amount into format and the amount. link
And in 4.6.4, It does not. link
More details here : fabric8io/kubernetes-client@c370d26

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dongjoon-hyun, a gentle ping.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ScrapCodes ScrapCodes changed the title [WIP][SPARK-30715][k8s][FOLLOW-UP] Update k8s client version in integratio… [SPARK-30715][k8s][FOLLOW-UP] Update k8s client version in integratio… Mar 19, 2020
@dongjoon-hyun dongjoon-hyun changed the title [SPARK-30715][k8s][FOLLOW-UP] Update k8s client version in integratio… [SPARK-30715][k8s][FOLLOW-UP] Update k8s client version in IT as well Mar 22, 2020
@dongjoon-hyun dongjoon-hyun changed the title [SPARK-30715][k8s][FOLLOW-UP] Update k8s client version in IT as well [SPARK-30715][K8S][TESTS][FOLLOWUP] Update k8s client version in IT as well Mar 22, 2020
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Merged to master.
Thank you, @ScrapCodes .

@ScrapCodes ScrapCodes deleted the follow-up-spark-30715 branch March 31, 2020 07:45
sjincho pushed a commit to sjincho/spark that referenced this pull request Apr 15, 2020
…s well

### What changes were proposed in this pull request?
This is a follow up for SPARK-30715 . Kubernetes client version in sync in integration-tests and kubernetes/core

### Why are the changes needed?
More than once, the kubernetes client version has gone out of sync between integration tests and kubernetes/core. So brought them up in sync again and added a comment to save us from future need of this additional followup.

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Manually.

Closes apache#27948 from ScrapCodes/follow-up-spark-30715.

Authored-by: Prashant Sharma <prashsh1@in.ibm.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun
Copy link
Member

Hi.
I'll backport this to branch-3.0 to address the recent JDK8 issue on K8s 1.17.x.

dongjoon-hyun pushed a commit that referenced this pull request May 24, 2020
…s well

### What changes were proposed in this pull request?
This is a follow up for SPARK-30715 . Kubernetes client version in sync in integration-tests and kubernetes/core

### Why are the changes needed?
More than once, the kubernetes client version has gone out of sync between integration tests and kubernetes/core. So brought them up in sync again and added a comment to save us from future need of this additional followup.

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Manually.

Closes #27948 from ScrapCodes/follow-up-spark-30715.

Authored-by: Prashant Sharma <prashsh1@in.ibm.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 3799d2b)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants