[Packaging] BREAKING CHANGE: Remove unnecessary packages in docker image #27567
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
️✔️AzureCLI-FullTest
|
azure-client-tools-bot-prd
bot
added this to the October 2023 (2023-11-15)❗ Breaking change release ❗ milestone
️✔️AzureCLI-BreakingChangeTest
|
microsoft-github-policy-service
bot
requested review from
jiasli,
wangzelin007 and
yonzhan
|
reduce docker image size |
bebound
force-pushed
the
remove-unused
branch
2 times, most recently
from
4ea8e41
to
80389ec
Compare
bebound
force-pushed
the
remove-unused
branch
from
80389ec
to
5516ed5
Compare
bebound
changed the title
bebound
commented
Oct 11, 2023
Dockerfile
Outdated
| | xargs -r apk info --installed \ | ||
| | sort -u \ | ||
| )" \ | ||
| && apk add --virtual .rundeps $runDeps |
There was a problem hiding this comment.
.rundeps makes no sense, see #27269 (comment)
bebound
commented
Oct 11, 2023
Dockerfile
Outdated
| # ca-certificates bash bash-completion - for convenience | ||
| # libintl icu-libs - required by azure-devops https://github.com/Azure/azure-cli/pull/9683 | ||
| # libc6-compat - required by az storage blob sync https://github.com/Azure/azure-cli/issues/10381 | ||
| # gcc python3-dev musl-dev linux-headers libffi-dev - temporarily required by psutil | ||
|
|
||
| WORKDIR azure-cli | ||
| COPY . /azure-cli |
There was a problem hiding this comment.
This can be replaced by --mount to save another 100MB.
I'll create a new PR for this.
3 tasks
jiasli
reviewed
Oct 18, 2023
|
|
||
| # curl - required for installing jp | ||
| # jq - we include jq as a useful tool |
There was a problem hiding this comment.
jq is a very useful tool and we frequently see users using jq to parse Azure CLI's output. Removing tools like this will cause a breaking change.
jiasli
reviewed
Oct 18, 2023
|
|
||
| ARG JP_VERSION="0.2.1" | ||
|
|
||
| RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && curl -L https://github.com/jmespath/jp/releases/download/${JP_VERSION}/jp-linux-$arch -o /usr/local/bin/jp \ |
There was a problem hiding this comment.
jp is also demanded by some users. See the discussion at #25590.
There was a problem hiding this comment.
I added jp and jq again.
Should we also make ssh-keygen available by default? #1343
PS: Our docker image is more like a Linux image than a CLI image. The entrypoint is bash rather than az
jiasli
previously approved these changes
Oct 23, 2023
jiasli
reviewed
Oct 23, 2023
alpine.dockerfile
Outdated
| # ca-certificates bash bash-completion jq jp openssh-keygen - for convenience | ||
| # libintl icu-libs - required by azure-devops https://github.com/Azure/azure-cli/pull/9683 | ||
| # libc6-compat - required by az storage blob sync https://github.com/Azure/azure-cli/issues/10381 | ||
| # gcc python3-dev musl-dev linux-headers libffi-dev - temporarily required by psutil |
There was a problem hiding this comment.
python3-dev is not in the original alpine.dockerfile. Is it really needed?
There was a problem hiding this comment.
When install psutil in alpine, it prints
psutil could not be installed from sources because gcc is not installed. Try running:
sudo apk add gcc python3-dev
Weird, it can be built without python3-dev.
bebound
commented
Oct 23, 2023
|
|
||
| # We don't use openssl (3.0) for now. We only install it so that users can use it. | ||
| RUN apk add --no-cache bash openssh ca-certificates jq curl openssl perl git zip \ | ||
| && apk add --no-cache --virtual .build-deps gcc make openssl-dev libffi-dev musl-dev linux-headers \ |
There was a problem hiding this comment.
We used to remove .build-deps until #8291.
PS: az extension add -n azure-batch-cli-extensions works without those dependencies. I guess some dependencies finally provide the wheel packages over time.
Choosing Alpine as a base image is not a good idea, as some packages do not provide binary packages and the user must build them from source, such as the psutil package.
It's common to use cibuildwheel to build wheel. However, Alpine Linux uses musl, while manylinux wheels are based on glibc, so some packages are not available on Alpine. (Cibuildwheel supports build musllinux wheel, but not all package provide musl wheel)
Ref: pypa/pip#9735
jiasli
approved these changes
Oct 23, 2023
|
Very comprehensive history analysis. Nice work! |
This was referenced Nov 8, 2023
Closed
rguptar
added a commit
to Azure/InnovationEngine
that referenced
this pull request
Dec 6, 2023
Multiple packages were removed as part of this breaking change. Azure/azure-cli#27567
github-merge-queue bot
pushed a commit
to Azure/InnovationEngine
that referenced
this pull request
Dec 6, 2023
Multiple packages were removed as part of this breaking change. Azure/azure-cli#27567
This was referenced Jan 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Remove unnecessary packages from the image to reduce its size from 1122MB to 922MB.
The reason for removing it is not only to reduce the size but also to comply with Docker's best practices.
Removed packages:
openssh curl openssl perl git zip gcc make openssl-dev libffi-dev musl-dev linux-headersNote: After this PR,
connectedk8s, hybridaks, k8s-configuration, rdbms-connect, mlextensions fail to install bacause of the lack of musllinux wheel. Please install the related dev packages withapk add xxxand try again.This checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.