Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let build_server.sh take whl file URL as an input argument. #5206

Merged
merged 1 commit into from Oct 26, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
39 changes: 31 additions & 8 deletions tensorflow/tools/dist_test/build_server.sh
Expand Up @@ -16,7 +16,14 @@
#
# Builds the test server for distributed (GRPC) TensorFlow
#
# Usage: build_server.sh <docker_image_name> [--test]
# Usage: build_server.sh <docker_image_name> <whl_url> [--test]
#
# Arguments:
# docker_image_name: Name of the docker image to build.
# E.g.: tensorflow/tf_grpc_test_server:0.11.0rc1
#
# whl_url: URL from which the TensorFlow whl file will be downloaded.
# E.g.: https://ci.tensorflow.org/view/Nightly/job/nightly-matrix-cpu/TF_BUILD_IS_OPT=OPT,TF_BUILD_IS_PIP=PIP,TF_BUILD_PYTHON_VERSION=PYTHON2,label=cpu-slave/lastSuccessfulBuild/artifact/pip_test/whl/tensorflow-0.11.0rc1-cp27-none-linux_x86_64.whl
#
# The optional flag --test lets the script to use the Dockerfile for the
# testing GRPC server. Without the flag, the script will build the non-test
Expand All @@ -33,28 +40,44 @@ die() {
}

# Check arguments
if [[ $# != 1 ]] && [[ $# != 2 ]]; then
die "Usage: $0 <docker_image_name> [--test]"
if [[ $# -lt 2 ]]; then
die "Usage: $0 <docker_image_name> <whl_url> [--test]"
fi

DOCKER_IMG_NAME=$1
shift
WHL_URL=$2
shift 2

# Current script directory
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

DOCKER_FILE="${DIR}/server/Dockerfile"
BUILD_DIR=$(mktemp -d)
echo ""
echo "Using whl file URL: ${WHL_URL}"
echo "Building in temporary directory: ${BUILD_DIR}"

cp -r ${DIR}/* "${BUILD_DIR}"/ || \
die "Failed to copy files to ${BUILD_DIR}"

DOCKER_FILE="${BUILD_DIR}/server/Dockerfile"
if [[ $1 == "--test" ]]; then
DOCKER_FILE="${DIR}/server/Dockerfile.test"
DOCKER_FILE="${BUILD_DIR}/server/Dockerfile.test"
fi
echo "Using Docker file: ${DOCKER_FILE}"

# Download whl file into the build context directory.
wget -P "${BUILD_DIR}" ${WHL_URL} || \
die "Failed to download tensorflow whl file from URL: ${WHL_URL}"

if [[ ! -f "${DOCKER_FILE}" ]]; then
die "ERROR: Unable to find dockerfile: ${DOCKER_FILE}"
fi
echo "Dockerfile: ${DOCKER_FILE}"

# Call docker build
docker build --no-cache -t "${DOCKER_IMG_NAME}" \
-f "${DOCKER_FILE}" \
"${DIR}"
-f "${DOCKER_FILE}" "${BUILD_DIR}" || \
die "Failed to build docker image: ${DOCKER_IMG_NAME}"

# Clean up docker build context directory.
rm -rf "${BUILD_DIR}"
7 changes: 4 additions & 3 deletions tensorflow/tools/dist_test/server/Dockerfile
Expand Up @@ -34,9 +34,10 @@ RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
python get-pip.py && \
rm get-pip.py

# Install TensorFlow CPU version from nightly build
RUN pip --no-cache-dir install \
https://ci.tensorflow.org/view/Nightly/job/nightly-matrix-cpu/TF_BUILD_IS_OPT=OPT,TF_BUILD_IS_PIP=PIP,TF_BUILD_PYTHON_VERSION=PYTHON2,label=cpu-slave/lastSuccessfulBuild/artifact/pip_test/whl/tensorflow-0.11.0rc1-cp27-none-linux_x86_64.whl
# Install TensorFlow wheel
COPY tensorflow-*.whl /
RUN pip install /tensorflow-*.whl && \
rm -f /tensorflow-*.whl

# Copy files, including the GRPC server binary at
# server/grpc_tensorflow_server.py
Expand Down
7 changes: 4 additions & 3 deletions tensorflow/tools/dist_test/server/Dockerfile.test
Expand Up @@ -40,9 +40,10 @@ RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
# Install python panda for the census wide&deep test
RUN pip install --upgrade pandas==0.18.1

# Install TensorFlow CPU version.
RUN pip --no-cache-dir install \
https://ci.tensorflow.org/view/Nightly/job/nightly-matrix-cpu/TF_BUILD_IS_OPT=OPT,TF_BUILD_IS_PIP=PIP,TF_BUILD_PYTHON_VERSION=PYTHON2,label=cpu-slave/lastSuccessfulBuild/artifact/pip_test/whl/tensorflow-0.11.0rc1-cp27-none-linux_x86_64.whl
# Install TensorFlow wheel
COPY tensorflow-*.whl /
RUN pip install /tensorflow-*.whl && \
rm -f /tensorflow-*.whl

# Copy files, including the GRPC server binary at
# server/grpc_tensorflow_server.py
Expand Down