Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

updated dockerfiles for grpc builds for powerPC compilation~ #2262

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

updated dockerfiles for grpc builds for powerPC compilation~ #2262

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 5 additions & 2 deletions cmd/suggestion/hyperband/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -5,9 +5,12 @@ ENV TARGET_DIR /opt/katib
ENV SUGGESTION_DIR cmd/suggestion/hyperband/v1beta1
ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python

RUN if [ "${TARGETARCH}" = "ppc64le" ] || [ "${TARGETARCH}" = "arm64" ]; then \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reasons to remove arm64 from here?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No specific reason but I think its typo and we can fix it by nested if-else condition.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I couldn't follow it. where is a typo?
Could you clarify it? Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, these packages are required to support arm64, isn't ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We verify grpcio installation only for ppc64le but not for arm64 and while raising the PR, I think arm64 has been mistakenly removed in hyperband dockerfile. So will update dockerfile and add a condition where if it is ppc64le then only install grpcio from source...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We verify grpcio installation only for ppc64le but not for arm64 and while raising the PR, I think arm64 has been mistakenly removed in hyperband dockerfile. So will update dockerfile and add a condition where if it is ppc64le then only install grpcio from source...

Ah, I see. Thanks.

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
Comment on lines +11 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any other approach using a package manager like apt?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm checking alternate approach for grpcio installation..once done will add it here.Thanks

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tenzen-y Yes we can install grpcio by setting flag GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 to pip install and have verified all the dockerfiles with the same changes. I will update the PR.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need a git clone here, you can just set this flag GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 down in the pip install block to ensure pip source building doesn't pick up the BoringSSL library.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkumatag Thanks for your response and yes I was able to verify the grpc installation by setting the flag GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1. I will update PR with the changes.

apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down
5 changes: 4 additions & 1 deletion cmd/suggestion/hyperopt/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -7,7 +7,10 @@ ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TAR

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down
5 changes: 4 additions & 1 deletion cmd/suggestion/nas/darts/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -7,7 +7,10 @@ ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TAR

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down
5 changes: 4 additions & 1 deletion cmd/suggestion/optuna/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -7,7 +7,10 @@ ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TAR

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down
5 changes: 4 additions & 1 deletion cmd/suggestion/pbt/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -7,7 +7,10 @@ ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TAR

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down
5 changes: 4 additions & 1 deletion cmd/suggestion/skopt/v1beta1/Dockerfile
View file
Open in desktop
Expand Up @@ -7,7 +7,10 @@ ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TAR

RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
apt-get -y install gfortran libopenblas-dev liblapack-dev libssl-dev gcc g++ pkg-config git && \
export REPO_ROOT=grpc && git clone -b v1.60.1 https://github.com/grpc/grpc $REPO_ROOT && \
cd $REPO_ROOT && git submodule update --init && \
pip install -r requirements.txt && GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 pip install . && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*; \
fi
Expand Down