Skip to content

Conversation

jordanstephens
Copy link
Member

@jordanstephens jordanstephens commented Jun 30, 2025

@commit111 and @lionello reported that the docker image size for the docs chatbot was over 3GB on June 27. Unfortunately, we had not had a successful image build since May 22, so many changes have been made in that timeframe. Something in abbcd8a...c42f4cb inflated the image size from ~1.5GB to 3GB.

Looking into the two images with dive, I discovered that it there was a new 2.7GB nvidia package in the larger June 27 image which wasn't present in the smaller May 22 image.

Looking at the compose.yaml file, we do not specify gpu reservation, so the pytorch workloads are certainly running on cpu, and we shouldn't need any nvidia packages. Pytorch offers pre-built cpu-only wheels, and that would remove the need to download the unused nvidia package. This pre-built wheel is only available for x86, which is fine in production, since it looks like we are already deploying to x86, but since we are developing on aarch macs, we need to specify the platform in the compose file to prevent docker from building for the local architecture.

While we're at it, i'm removing any options to run on cpu, since we are definitely always running on cpu already.

Here's the dive output for the 3GB June 27 image:

larger-image

When I build locally, I get an image that is ~1.5GB:

docker image ls | head
REPOSITORY                                                                                            TAG                           IMAGE ID       CREATED         SIZE
docs-chatbot-app                                                                                      latest                        2209e78e161a   5 minutes ago   1.56GB

only available for x86
as we will now always run on cpu--these are irrelevant.
@nullfunc nullfunc merged commit d65b548 into main Jul 11, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants