Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build arm images #1305

Merged
merged 10 commits into from
Nov 4, 2022
Merged

Build arm images #1305

merged 10 commits into from
Nov 4, 2022

Conversation

andrewwdye
Copy link
Contributor

TL;DR

Build arm64 images for flytekit and sqlalchemy

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

This change uses the docker/build-push-action and docker buildx multi-arch support to build images for amd64 and arm64 across all supported python versions.

Verified images successfully build in forked repo with mock releases. See publish workflow and images.

Note: Looks like there is a dependency regression in master. Attempting to run tasks with these images fails with

Message: 
local/lib/python3.10/site-packages/flytekitplugins/fsspec/persist.py", line 95, in get
    fs = self.get_filesystem(from_path)
  File "/usr/local/lib/python3.10/site-packages/flytekitplugins/fsspec/persist.py", line 64, in get_filesystem
    return fsspec.filesystem(protocol, **kwargs)  # type: ignore
  File "/usr/local/lib/python3.10/site-packages/fsspec/registry.py", line 283, in filesystem
    cls = get_filesystem_class(protocol)
  File "/usr/local/lib/python3.10/site-packages/fsspec/registry.py", line 245, in get_filesystem_class
    raise ImportError(bit["err"]) from e
ImportError: Install s3fs to access S3

I verified an amd64 image built from master is missing s3*

docker build -f Dockerfile --build-arg PYTHON_VERSION=3.7 --build-arg VERSION=1.2.3 --build-arg DOCKER_IMAGE=foo --platform=linux/amd64 .

docker run --rm -it 046bebaf828a pip list | grep s3
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

Tracking Issue

NA

Follow-up issue

NA

Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
@codecov
Copy link

codecov bot commented Nov 4, 2022

Codecov Report

Merging #1305 (b397a0d) into master (cb651e8) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1305   +/-   ##
=======================================
  Coverage   68.87%   68.87%           
=======================================
  Files         287      287           
  Lines       26429    26429           
  Branches     2495     2495           
=======================================
  Hits        18202    18202           
  Misses       7743     7743           
  Partials      484      484           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Dockerfile Outdated Show resolved Hide resolved
Signed-off-by: Andrew Dye <andrewwdye@gmail.com>
@wild-endeavor wild-endeavor merged commit 3ffb6df into flyteorg:master Nov 4, 2022
@@ -10,11 +10,14 @@ ENV PYTHONPATH /root
ARG VERSION
ARG DOCKER_IMAGE

RUN apt-get update && apt-get install build-essential -y
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could bring the builder back in a later PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants