Skip to content

Minimize sqlflow:jupyter image size #2552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 28, 2020
Merged

Minimize sqlflow:jupyter image size #2552

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
efdeeef
Minimize sqlflow:mysql image
lhw362950217 Jun 23, 2020
58c8edc
fix ci
lhw362950217 Jun 24, 2020
3164e02
Merge https://github.com/sql-machine-learning/sqlflow into minimize_d…
lhw362950217 Jun 28, 2020
27350d3
minimize jupyter image size
lhw362950217 Jun 28, 2020
fda415a
modify doc
lhw362950217 Jun 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 12 additions & 21 deletions docker/jupyter/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,23 @@
# This Dockerfile containers Jupyter Notebook server with many
# SQLFlow tutorials and SQLFlow magic command.

FROM jupyter/base-notebook
FROM alpine:3.12

# using root user to avoid permission deni
USER root
# Choose fastest mirrors for apt-get and pip
COPY docker/dev/find_fastest_resources.sh /usr/local/bin/find_fastest_resources.sh
RUN /bin/bash -c 'source find_fastest_resources.sh \
&& echo "Choose the fastest APT source ..." \
&& choose_fastest_apt_source \
&& echo "Choose the fastest PIP source ..." \
&& choose_fastest_pip_source'
COPY docker/dev/find_fastest_resources.sh /usr/local/bin/
RUN /bin/sh -c "source find_fastest_resources.sh && \
choose_fastest_alpine_source && \
choose_fastest_pip_source"

# Install dependencies.
COPY docker/jupyter/js /jupyter/js

RUN apt-get -qq update

COPY docker/jupyter/install-jupyter.bash /jupyter
RUN /jupyter/install-jupyter.bash

# switch back to the default user to avoid accidental container runs as root,
# this env comes from base Dockerfile: https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile#L13
USER $NB_UID
RUN apk add --no-cache python3 py3-pip py3-pyzmq py3-grpcio

# Install IPythono Notebook tutorials
COPY /docker/jupyter/js /jupyter/js/
COPY build/tutorial /workspace
COPY docker/jupyter/install-jupyter.sh /jupyter/install-jupyter.sh
RUN /bin/sh /jupyter/install-jupyter.sh

# Cleanup
RUN apk del --purge py3-pip

# The following SQLFlow gRPC server endpoint implies the server runs in a container,
# and if container has the option --net=container:sqlflow_server_container, SQLFlow magic
Expand Down
18 changes: 12 additions & 6 deletions docker/jupyter/install-jupyter.bash → docker/jupyter/install-jupyter.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/bin/sh

# Copyright 2020 The SQLFlow Authors. All rights reserved.
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -15,12 +15,18 @@

set -e

# This file depends on install-python.bash.
# install jupyterhub Python package so that this image can be used as jupyterhub
# singleuser notebook server, ref: https://github.com/jupyterhub/jupyterhub/tree/master/singleuser
pip install --quiet \
jupyterhub==1.1.0 \
Copy link
Collaborator

@Yancey0623 Yancey0623 Jun 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jupyterhub Python package is necessary to work with jupyterhub service.

sqlflow==0.10.0 # sqlflow is the Python client of SQLFlow server.
# Install pandas pre-compiled apk, we do not want to build
# this python package locally because it relies on gcc and
# other build tools, which make the image very large
wget -q http://cdn.sqlflow.tech/alpine/py3-pandas-1.0.3-r0.apk
wget -q -P /etc/apk/keys/ http://cdn.sqlflow.tech/alpine/sqlflow-5ef80180.rsa.pub
apk add py3-pandas-1.0.3-r0.apk && rm py3-pandas-1.0.3-r0.apk

pip -q install \
notebook \
sqlflow==0.10.0

# Load SQLFlow's Jupyter magic command
# automatically. c.f. https://stackoverflow.com/a/32683001.
Expand All @@ -33,7 +39,7 @@ mkdir -p /workspace
>> "$IPYTHON_STARTUP"/00-first.py

# Enable highlighting, see https://stackoverflow.com/questions/43641362
NOTEBOOK_DIR=$(python -c "print(__import__('notebook').__path__[0])")
NOTEBOOK_DIR=$(python3 -c "print(__import__('notebook').__path__[0])")
CODE_MIRROR_MODE_PATH=$NOTEBOOK_DIR/static/components/codemirror/mode
mkdir -p "$HOME"/.jupyter/custom/
mkdir -p "$CODE_MIRROR_MODE_PATH"/sqlflow
Expand Down