Docker performance improvements and reduce image size

[k1lgor]

Info

• Refactored Dockerfile in multi-stage build - helps to build faster and reduces the image size

  Time for initial build:

  • [+] Building 113.2s (15/15) FINISHED

• Refactored entrypoint.sh - it is more optimized and avoids unnecessary iterations

[definitiontv]

Pretty sure this will break ARM compatibility

[definitiontv]

Pretty sure this will break ARM compatibility

Belay - that - compiles fine on my ARM. New size is less than 1/2 the original - nice!!!

[ATheorell]

Thanks for this! Can you have a look at this too @TheoMcCabe ?

[AntonOsika]

I don't like the maxdepth. Why do we have that?

Otherwise, do you see any risks with this PR @k1lgor ?

[AntonOsika]

Curious what was approx image size before this? :)

[k1lgor]

I don't like the maxdepth. Why do we have that?

Otherwise, do you see any risks with this PR @k1lgor ?

@AntonOsika
Restricts the search to only the immediate children of the directory, and avoids recursive search.

[k1lgor]

Curious what was approx image size before this? :)

60% more or less

[definitiontv]

Pretty sure this will break ARM compatibility

Belay - that - compiles fine on my ARM. New size is less than 1/2 the original - nice!!!

I may have spoken too soon. I only quickly ran but there are some problems with using alpine for those of us who use docker compose containers.
1:/ ASH instead of default BASH - even adding it to the APK ADD didn't seem to work for me - so I get a binary error on ARM - not sure why.
2:/ Relatexd to 1:/. I I actually always run gpte inside the container and link to project directory - most of the code gererated on my system assumes python, bash and APT available for installing packages. Alpine has significant differences which need to be "explained"
3:/ Should be a seperate issue but the default example project directory inside the git repo should NOT really be persisted for real work- really should use as default root the USERs home folder or path psssed or ENVIRONMENT and that avoids the very dangerous permission hack whch is currently being used. Many of us NEVER store new unrelated code or data in external GIT REPOS.
4:/ This project continues the default of running the example project at startup. That's a costly proof of concept which shoudl not be needed now? I overide this in my compose file but for those running direct from image I think the default should be simply the project folder passed by the user or a prompt?

AM I missing something? are other people succesfully running it all like this in their workflow??

[k1lgor]

@definitiontv Maybe share some error msg, so I will be able to find a solution.

Those are the supported arch for alpine described in Docker Hub

[k1lgor]

Hi all,
The example below uses the same Ubuntu image. With this example, the reduced size is about 50% and the initial build time was less that 2 minutes. [+] Building 113.4s (14/14) FINISHED

# Use a lightweight base image
FROM python:3.11-slim AS builder
# Install necessary system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends tk tcl gcc curl
# Set working directory
WORKDIR /app
# Copy source code and entrypoint script
COPY . .
COPY docker/entrypoint.sh ./entrypoint.sh
# Install dependencies and application in a virtual environment
RUN python -m venv /venv && /venv/bin/pip install --upgrade pip && /venv/bin/pip install .
# Final image
FROM python:3.11-slim
# Set working directory
WORKDIR /app
# Copy application and virtual environment from the builder stage
COPY --from=builder /app /app
COPY --from=builder /venv /venv
# Add the virtual environment to the PATH
ENV PATH="/venv/bin:$PATH"
# Set the entrypoint
ENTRYPOINT ["bash", "entrypoint.sh"]

Please take a look and be more active on the PR @definitiontv @AntonOsika @ATheorell @TheoMcCabe

[AntonOsika]

Merging!

[k1lgor]

If multiple users face issues with Alpine, we can easily switch back to Ubuntu with almost the same reduction in image size.

[snf]

@k1lgor it looks like something depends on arrow which doesn't have wheels for Alpine. It tries to build it unsuccesfully because it's missing dependencies like cmake, make, gcc and arrow libraries. Didn't get very far debugging the issue but went back to the ubuntu image which works fine with the current pip dependencies.

Possibly related: apache/arrow#18036

[k1lgor]

I will take a look at this one.

[viborc]

Hey @k1lgor - any updates or expected actions on this one?

[k1lgor]

@k1lgor it looks like something depends on arrow which doesn't have wheels for Alpine. It tries to build it unsuccesfully because it's missing dependencies like cmake, make, gcc and arrow libraries. Didn't get very far debugging the issue but went back to the ubuntu image which works fine with the current pip dependencies.

Possibly related: apache/arrow#18036

@viborc
The Alpine image does not come with those dependencies pre-installed. Pyarrow comes without a wheel and needs to be installed from source, but I have not had any luck with builds up until this point.