-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable pyspark console in docker container #20
Conversation
…ove alpine/apk dependencies at the end of the build to reduce image size. README: describe how to start pyspark console inside Docker container.
git \ | ||
wget | ||
wget \ | ||
&& apk add --update python |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason this line can't just be python
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both git
and wget
are installed into the virtual package build-dependencies
, which gets deleted at the end of the build. To keep python
out of this package, a separate add
without --virtual
is required.
Actually, --update
is not required again here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, gotcha!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(2/2) Installing python2 (2.7.16-r2)
We should go with Python 3 here. Any reason we're using Python 2?
I tried python 3 first but pyspark (?) requires version 2. I also found
that strange, but didn't investigate further.
…On Tue, Feb 25, 2020, 13:35 Nick Ruest ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In Dockerfile
<#20 (comment)>
:
> git \
- wget
+ wget \
+ && apk add --update python
(2/2) Installing python2 (2.7.16-r2)
We should go with Python 3 here. Any reason we're using Python 2?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#20?email_source=notifications&email_token=AAADUQZ2RDUZNYJABC7OUWDREUGAVA5CNFSM4K2LNI4KYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCWZ5DGA#discussion_r383851252>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADUQ7O2QLJ5RUKQ5LQ543REUGAVANCNFSM4K2LNI4A>
.
|
@sepastian I don't believe so. I use Python 3.7.3 locally with PySpark, and |
Ok, will take another look and update the PR, python 3 should be used.
…On Tue, Feb 25, 2020, 13:48 Nick Ruest ***@***.***> wrote:
@sepastian <https://github.com/sepastian> I don't believe so. I use
Python 3.7.3 locally with PySpark, and aut.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#20?email_source=notifications&email_token=AAADUQZO327U4BA2WRYZTV3REUHR7A5CNFSM4K2LNI4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM32PNQ#issuecomment-590849974>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADUQ7XXUIX2JOVRNGHESTREUHR7ANCNFSM4K2LNI4A>
.
|
Closing this and creating a new PR against current docker-aut. |
Additions:
Dockerfile