Skip to content

[SPARK-56889][INFRA][FOLLOWUP] Install python3-pyparsing before add-apt-repository#55956

Closed
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-56889-2
Closed

[SPARK-56889][INFRA][FOLLOWUP] Install python3-pyparsing before add-apt-repository#55956
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-56889-2

Conversation

@dongjoon-hyun
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun commented May 18, 2026

What changes were proposed in this pull request?

This PR aims to install python3-pyparsing before invoking add-apt-repository.

Why are the changes needed?

To avoid ModuleNotFoundError: No module named 'pyparsing' error inside add-apt-repository.

#19 [13/32] RUN add-apt-repository ppa:deadsnakes/ppa
#19 0.509 Traceback (most recent call last):
#19 0.509   File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 34, in <module>
#19 0.509     from httplib2 import proxy_info_from_environment
#19 0.509   File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 52, in <module>
#19 0.509     from . import auth
#19 0.509   File "/usr/lib/python3/dist-packages/httplib2/auth.py", line 4, in <module>
#19 0.509     import pyparsing as pp
#19 0.509 ModuleNotFoundError: No module named 'pyparsing'

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual review.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun
Copy link
Copy Markdown
Member Author

cc @zhengruifeng , @LuciferYang , @peter-toth

Copy link
Copy Markdown
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if this can be fixed fundamentally.

@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Thank you, @LuciferYang . Since this is a CI issue, let me merge to validate this.

We can still keep @zhengruifeng 's PR as a fallback.

dongjoon-hyun added a commit that referenced this pull request May 18, 2026
…d-apt-repository`

### What changes were proposed in this pull request?

This PR aims to install `python3-pyparsing` before invoking `add-apt-repository`.

### Why are the changes needed?

To avoid `ModuleNotFoundError: No module named 'pyparsing'` error inside `add-apt-repository`.

```
#19 [13/32] RUN add-apt-repository ppa:deadsnakes/ppa
#19 0.509 Traceback (most recent call last):
#19 0.509   File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 34, in <module>
#19 0.509     from httplib2 import proxy_info_from_environment
#19 0.509   File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 52, in <module>
#19 0.509     from . import auth
#19 0.509   File "/usr/lib/python3/dist-packages/httplib2/auth.py", line 4, in <module>
#19 0.509     import pyparsing as pp
#19 0.509 ModuleNotFoundError: No module named 'pyparsing'
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual review.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #55956 from dongjoon-hyun/SPARK-56889-2.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit c00f9a8)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Merged to master/4.x.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-56889-2 branch May 18, 2026 16:12
@dongjoon-hyun
Copy link
Copy Markdown
Member Author

Oh, unfortunately, it seems to fail still after installing python3-pyparsing.

#9 [ 3/32] RUN apt-get update && apt-get install -y     build-essential     ca-certificates     curl     gfortran     git     gnupg     libcurl4-openssl-dev     libfontconfig1-dev     libfreetype6-dev     libfribidi-dev     libgit2-dev     libharfbuzz-dev     libjpeg-dev     liblapack-dev     libopenblas-dev     libpng-dev     libpython3-dev     libssl-dev     libtiff5-dev     libuv1-dev     libwebp-dev     libxml2-dev     nodejs     npm     openjdk-17-jdk-headless     pandoc     pkg-config     python3.11     python3-psutil     python3-pyparsing     qpdf     r-base     ruby     ruby-dev     software-properties-common     wget     zlib1g-dev     && rm -rf /var/lib/apt/lists/*
...
#19 [13/32] RUN add-apt-repository ppa:deadsnakes/ppa
#19 0.525 Traceback (most recent call last):
#19 0.525   File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 34, in <module>
#19 0.525     from httplib2 import proxy_info_from_environment
#19 0.525   File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 52, in <module>
#19 0.525     from . import auth
#19 0.525   File "/usr/lib/python3/dist-packages/httplib2/auth.py", line 4, in <module>
#19 0.525     import pyparsing as pp
#19 0.525 ModuleNotFoundError: No module named 'pyparsing'

Let me merge @zhengruifeng 's PR.

zhengruifeng pushed a commit that referenced this pull request May 19, 2026
…ng from dockerfile

### What changes were proposed in this pull request?

Remove `python3-pyparsing` from `dev/infra/Dockerfile`.

### Why are the changes needed?

This line was added in #55956 as an attempt to fix the image build issue caused by removing 3.10.

This was not the correct fix. The reason CI failed was because we somehow uninstalled `pyparsing` from system python3. We mitigated the issue later in #55946 . We should not keep this line around - it could confuse people in the future.

This specific dockerfile is only used by 3.5. We should just retire the file after 3.5 EOL. We don't have to upgrade the 3.10 part (we didn't touch the 3.9 part of the file). We can just keep it as it is and delete it in the future. As we are using the docker file for 3.5, we probably shouldn't remove 3.10 because it's actually supported in 3.5.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

CI should work.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #55965 from gaogaotiantian/remove-pyparsing.

Authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
zhengruifeng pushed a commit that referenced this pull request May 19, 2026
…ng from dockerfile

### What changes were proposed in this pull request?

Remove `python3-pyparsing` from `dev/infra/Dockerfile`.

### Why are the changes needed?

This line was added in #55956 as an attempt to fix the image build issue caused by removing 3.10.

This was not the correct fix. The reason CI failed was because we somehow uninstalled `pyparsing` from system python3. We mitigated the issue later in #55946 . We should not keep this line around - it could confuse people in the future.

This specific dockerfile is only used by 3.5. We should just retire the file after 3.5 EOL. We don't have to upgrade the 3.10 part (we didn't touch the 3.9 part of the file). We can just keep it as it is and delete it in the future. As we are using the docker file for 3.5, we probably shouldn't remove 3.10 because it's actually supported in 3.5.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

CI should work.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #55965 from gaogaotiantian/remove-pyparsing.

Authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
(cherry picked from commit eeecea4)
Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants