-
Notifications
You must be signed in to change notification settings - Fork 7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
emailservice Image Optimization #51
emailservice Image Optimization #51
Conversation
Merge from Upstream
Merge from master
…builds and alpine image
src/emailservice/Dockerfile
Outdated
|
||
# Install grpcio | ||
RUN python -m pip install --upgrade pip | ||
RUN pip install grpcio==${GRPC_PYTHON_VERSION} grpcio-tools==${GRPC_PYTHON_VERSION} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend we just put these two inside requirements.txt. Not sure why we need to treat them any differently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can do. I originally left them in there to mimic the gRPC docker image
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should just move it to requirements.txt.
Also I recommend keeping the original grpc version. The 1.15 has introduced new methods in the health.proto that we haven't implemented (not sure if it's a problem in Python, but it broke the app in Go).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to requirements.txt & rolled versions back to 1.12.1
src/emailservice/Dockerfile
Outdated
RUN pip3 install -r requirements.txt | ||
|
||
# Grab packages from builder | ||
COPY --from=builder /usr/local/lib/python2.7/ /usr/local/lib/python2.7/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a legit thing to do? Seemed a bit scary to me :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've seen some articles that recommend doing either that or copying the cache over. The goal was to get rid of the dependencies gRPC needs at compile time but not at run time (linux-headers
etc.)
src/emailservice/Dockerfile
Outdated
@@ -1,27 +1,52 @@ | |||
FROM python:2.7-alpine3.8 as base |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great if we could keep it python3.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Will move back to Python 3
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have moved everything to python 3
…s to their older versions (1.12.1, 1.14.1, and 1.12.1 respectively)
Latest image size is now ~270MB, which is to be expected as the python 3 docker images are ~20MB larger than the 2.7 images (but both are still smaller than the grpc/python image) From my terminal:
|
src/emailservice/requirements.txt
Outdated
grpcio-health-checking==1.15.0 | ||
grpcio-tools==1.15.0 | ||
grpcio==1.12.1 | ||
grpcio-health-checking==1.14.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ideally this also should be 1.12.1 but it's probably identical to 1.14.1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. The previous requirements.txt
had it at 1.14.1
so I kept it at that. I'll downgrade it and test
src/emailservice/Dockerfile
Outdated
@@ -1,32 +1,25 @@ | |||
FROM python:2.7-alpine3.8 as base | |||
FROM python:3.7.0-alpine3.8 as base |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend doing python:3-alpine
(or :3.7-alpine
) here so that:
- we get python 3.x patches
- we get stable alpine patches (not edge)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do.
src/emailservice/Dockerfile
Outdated
ENTRYPOINT [ "python", "-u", "email_server.py" ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend keeping this PYTHONUNBUFFERED=0 so that all processes (like pip
) benefit from it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Reduce docker image for emailservice to ~240 MB (down from ~ 1.31 GB) Main application (`email_server.py`) now runs as python 2.7. Before we had both Python 2.7 and Python 3 installed in the image. Switched to using `python:2.7-alpine3.8` as the base image, and used multi-stage dockerfiles to keep dependencies minimal. Fixes GoogleCloudPlatform#49 From my shell: ``` $ docker build -t emailservice:dev . && docker run -it emailservice:dev Sending build context to Docker daemon 97.28kB Step 1/17 : FROM python:2.7-alpine3.8 as base ---> b2bc7255b42c Step 2/17 : FROM base as builder ---> b2bc7255b42c Step 3/17 : RUN apk add --update --no-cache gcc linux-headers make musl-dev python-dev g++ cairo-dev cairo openssl-dev gobject-introspection-dev ---> Using cache ---> 6daf3d9fe49a Step 4/17 : ENV GRPC_PYTHON_VERSION 1.15.0 ---> Using cache ---> 3e33d97d9580 Step 5/17 : RUN python -m pip install --upgrade pip ---> Using cache ---> e8fa3879c282 Step 6/17 : RUN pip install grpcio==${GRPC_PYTHON_VERSION} grpcio-tools==${GRPC_PYTHON_VERSION} ---> Using cache ---> c6fba7743eed Step 7/17 : COPY requirements.txt . ---> Using cache ---> 1f6b0a444980 Step 8/17 : RUN pip install -r requirements.txt ---> Using cache ---> 8cc0a7af6aa8 Step 9/17 : FROM base as final ---> b2bc7255b42c Step 10/17 : RUN GRPC_HEALTH_PROBE_VERSION=v0.1.0-alpha.1 && wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && chmod +x /bin/grpc_health_probe ---> Using cache ---> e954a0384081 Step 11/17 : ENV PYTHONUNBUFFERED=0 ---> Using cache ---> 64ece3d72a66 Step 12/17 : WORKDIR /email_server ---> Using cache ---> 27b34dc14492 Step 13/17 : COPY --from=builder /usr/local/lib/python2.7/ /usr/local/lib/python2.7/ ---> Using cache ---> 60035ec8dfd4 Step 14/17 : RUN apk add --no-cache libstdc++ ---> Using cache ---> 920be90c126e Step 15/17 : COPY . . ---> Using cache ---> 9541bed2d7a0 Step 16/17 : EXPOSE 8080 ---> Using cache ---> 48fbeaa852b9 Step 17/17 : ENTRYPOINT [ "python", "email_server.py" ] ---> Using cache ---> ff317770992d Successfully built ff317770992d Successfully tagged emailservice:dev starting the email service in dummy mode. listening on port: 8080 ```
Reduce docker image for emailservice to ~240 MB (down from ~ 1.31 GB)
Main application (
email_server.py
) now runs as python 2.7. Before we had both Python 2.7 and Python 3 installed in the image.Switched to using
python:2.7-alpine3.8
as the base image, and used multi-stage dockerfiles to keep dependencies minimal.Fixes #49
From my shell: