Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-49005][K8S][3.4] Use 17-jammy tag instead of 17-jre to prevent Python 3.12 #47489

Closed

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jul 25, 2024

What changes were proposed in this pull request?

This PR aims to use 17-jammy tag instead of 17-jre to prevent Python 12.

Why are the changes needed?

Two days ago, eclipse-temurin:17 switched its baseline OS to Ubuntu 24.04 which brings Python 3.12.

$ docker run -it --rm eclipse-temurin:17-jre cat /etc/os-release | grep VERSION_ID
VERSION_ID="24.04"

$ docker run -it --rm eclipse-temurin:17-jammy cat /etc/os-release | grep VERSION_ID
VERSION_ID="22.04"

Since Python 3.12 supported is added only to Apache Spark 4.0.0, we need to keep using the previous OS, Ubuntu 22.04.

Does this PR introduce any user-facing change?

No. This aims to recover to the same OS for consistent behavior.

How was this patch tested?

Pass the CIs with K8s IT. Currently, it's broken at Python image building phase.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-49005][K8S][3.4] Use 17-jammy tag instead of 17 to prevent Python 3.12 [SPARK-49005][K8S][3.4] Use 17-jammy tag instead of 17-jre to prevent Python 3.12 Jul 25, 2024
@dongjoon-hyun
Copy link
Member Author

cc @yaooqinn and @viirya

@dongjoon-hyun
Copy link
Member Author

Also, cc @huaxingao

Copy link
Contributor

@huaxingao huaxingao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the PR @dongjoon-hyun

@dongjoon-hyun
Copy link
Member Author

Thank you, @huaxingao !

@dongjoon-hyun
Copy link
Member Author

Let me merge this to recover the CIs.

dongjoon-hyun added a commit that referenced this pull request Jul 25, 2024
…vent Python 3.12

### What changes were proposed in this pull request?

This PR aims to use `17-jammy` tag instead of `17-jre` to prevent Python 12.

### Why are the changes needed?

Two days ago, `eclipse-temurin:17` switched its baseline OS to `Ubuntu 24.04` which brings `Python 3.12`.

```
$ docker run -it --rm eclipse-temurin:17-jre cat /etc/os-release | grep VERSION_ID
VERSION_ID="24.04"

$ docker run -it --rm eclipse-temurin:17-jammy cat /etc/os-release | grep VERSION_ID
VERSION_ID="22.04"
```

Since Python 3.12 supported is added only to Apache Spark 4.0.0, we need to keep using the previous OS, `Ubuntu 22.04`.

- #43184
- #43192

### Does this PR introduce _any_ user-facing change?

No. This aims to recover to the same OS for consistent behavior.

### How was this patch tested?

Pass the CIs with K8s IT. Currently, it's broken at Python image building phase.

- https://github.com/apache/spark/actions/workflows/build_branch34.yml

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #47489 from dongjoon-hyun/SPARK-49005-3.4.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@dongjoon-hyun dongjoon-hyun deleted the SPARK-49005-3.4 branch July 25, 2024 17:43
Copy link
Member

@viirya viirya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@dongjoon-hyun
Copy link
Member Author

Thank you, @viirya !

@yaooqinn
Copy link
Member

Late LGTM

@dongjoon-hyun
Copy link
Member Author

Thank you, @yaooqinn .

szehon-ho pushed a commit to szehon-ho/spark that referenced this pull request Aug 7, 2024
…vent Python 3.12

This PR aims to use `17-jammy` tag instead of `17-jre` to prevent Python 12.

Two days ago, `eclipse-temurin:17` switched its baseline OS to `Ubuntu 24.04` which brings `Python 3.12`.

```
$ docker run -it --rm eclipse-temurin:17-jre cat /etc/os-release | grep VERSION_ID
VERSION_ID="24.04"

$ docker run -it --rm eclipse-temurin:17-jammy cat /etc/os-release | grep VERSION_ID
VERSION_ID="22.04"
```

Since Python 3.12 supported is added only to Apache Spark 4.0.0, we need to keep using the previous OS, `Ubuntu 22.04`.

- apache#43184
- apache#43192

No. This aims to recover to the same OS for consistent behavior.

Pass the CIs with K8s IT. Currently, it's broken at Python image building phase.

- https://github.com/apache/spark/actions/workflows/build_branch34.yml

No.

Closes apache#47489 from dongjoon-hyun/SPARK-49005-3.4.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants