Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker changes to support M1 laptops #380

Merged
merged 4 commits into from
Aug 18, 2023
Merged

Docker changes to support M1 laptops #380

merged 4 commits into from
Aug 18, 2023

Conversation

obgibson
Copy link
Collaborator

Context:

The docker build would fail on an M1 laptop with error messages like

Building wheels for collected packages: psycopg2
30.95   Building wheel for psycopg2 (setup.py): started
33.66   Building wheel for psycopg2 (setup.py): finished with status 'error'
33.71   error: subprocess-exited-with-error
33.71   
33.71   × python setup.py bdist_wheel did not run successfully.
33.71   │ exit code: 1
33.71   ╰─> [33 lines of output]
33.71       running bdist_wheel
33.71       running build
33.71       running build_py
33.71       creating build
33.71       creating build/lib.linux-x86_64-cpython-311
33.71       creating build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/errorcodes.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/errors.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/extensions.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/__init__.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/sql.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/_range.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/tz.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/pool.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/_json.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/extras.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       copying lib/_ipaddress.py -> build/lib.linux-x86_64-cpython-311/psycopg2
33.71       running build_ext
33.71       building 'psycopg2._psycopg' extension
33.71       creating build/temp.linux-x86_64-cpython-311
33.71       creating build/temp.linux-x86_64-cpython-311/psycopg
33.71       gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC "-DPSYCOPG_VERSION=2.9.7 (dt dec pq3 ext lo64)" -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=150003 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/usr/local/include/python3.11 -I. -I/usr/include/postgresql -I/usr/include/postgresql/15/server -I/usr/include/libxml2 -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-cpython-311/psycopg/adapter_asis.o -Wdeclaration-after-statement

and

metaflow-service-ui_backend-1  | Traceback (most recent call last):
metaflow-service-ui_backend-1  |   File "/usr/local/bin/ui_backend_service", line 33, in <module>
metaflow-service-ui_backend-1  |     sys.exit(load_entry_point('metadata-service', 'console_scripts', 'ui_backend_service')())
metaflow-service-ui_backend-1  |   File "/usr/local/bin/ui_backend_service", line 25, in importlib_load_entry_point
metaflow-service-ui_backend-1  |     return next(matches).load()
metaflow-service-ui_backend-1  |   File "/usr/local/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 208, in load
metaflow-service-ui_backend-1  |     module = import_module(match.group('module'))
metaflow-service-ui_backend-1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
metaflow-service-ui_backend-1  |     return _bootstrap._gcd_import(name[level:], package, level)
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
metaflow-service-ui_backend-1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
metaflow-service-ui_backend-1  |   File "/root/services/ui_backend_service/ui_server.py", line 8, in <module>
metaflow-service-ui_backend-1  |     from services.utils import DBConfiguration, logging, ORIGIN_TO_ALLOW_CORS_FROM
metaflow-service-ui_backend-1  |   File "/root/services/utils/__init__.py", line 13, in <module>
metaflow-service-ui_backend-1  |     from packaging.version import Version, parse
metaflow-service-ui_backend-1  | ModuleNotFoundError: No module named 'packaging'

Fix:

Using the full version of python, rather than a slim one. The platform is also explicitly specified in the dev dockerfile.

@obgibson obgibson requested a review from saikonen August 16, 2023 21:02
@saikonen
Copy link
Collaborator

Performed some slight cleanup to the PR, as the scope for this is only to fix the development environment for ARM macs.

some extra context for the changes:

  • without specifying the target platform, installing goose as part of the build process on ARM macs fails.
  • for some reason, compiling psycopg2 works correctly on x86 with python slim images, but fails on ARM (even when specifying x86 as platform)

@saikonen saikonen merged commit 90f1922 into master Aug 18, 2023
6 checks passed
@saikonen saikonen deleted the chore/docker-m1 branch August 18, 2023 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants