Docker compose up producing errors

[ionflow]

Hello, was trying to run docker compose up from v0.0.40 and getting the following errors. I'm on a MacBook do I need a C compiler to build this? I have gcc installed through xcode. But this is running in docker so I'm not sure how to give docker the power. Any insights here?

 => CACHED [5/7] COPY poetry.lock pyproject.toml ./                                                                                                                                                0.0s
=> [6/7] COPY ./ ./                                                                                                                                                                               0.0s
=> ERROR [7/7] RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi                                                                                            5.7s
------                                                                                                                                                                                                  
> [7/7] RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi:                                                                                                       
#0 0.669 Skipping virtualenv creation, as specified in config file.
#0 0.955 Installing dependencies from lock file
#0 1.257 
#0 1.257 Package operations: 87 installs, 1 update, 0 removals
#0 1.257 
#0 1.257   • Installing packaging (23.0)
#0 1.257   • Installing pyasn1 (0.4.8)
#0 1.258   • Installing six (1.16.0)
#0 1.616   • Installing asttokens (2.2.1)
#0 1.617   • Installing cachetools (5.3.0)
#0 1.617   • Installing certifi (2022.12.7)
#0 1.618   • Installing charset-normalizer (3.1.0)
#0 1.619   • Installing executing (1.2.0)
#0 1.619   • Installing frozenlist (1.3.3)
#0 1.622   • Installing idna (3.4)
#0 1.622   • Installing marshmallow (3.19.0)
#0 1.623   • Installing multidict (6.0.4)
#0 1.688   • Installing mypy-extensions (1.0.0)
#0 1.689   • Installing parso (0.8.3)
#0 1.702   • Installing platformdirs (3.1.1)
#0 1.741   • Installing protobuf (4.22.1)
#0 1.801   • Installing ptyprocess (0.7.0)
#0 1.807   • Installing pure-eval (0.2.2)
#0 1.824   • Installing pyasn1-modules (0.2.8)
#0 1.829   • Installing pyparsing (3.0.9)
#0 1.876   • Installing rsa (4.9)
#0 1.894   • Installing sniffio (1.3.0)
#0 1.912   • Installing traitlets (5.9.0)
#0 1.918   • Installing typing-extensions (4.5.0)
#0 1.924   • Installing urllib3 (1.26.15)
#0 1.926   • Installing wcwidth (0.2.6)
#0 2.073   • Installing aiosignal (1.3.1)
#0 2.073   • Installing anyio (3.6.2)
#0 2.074   • Installing async-timeout (4.0.2)
#0 2.074   • Installing attrs (22.2.0)
#0 2.075   • Installing backcall (0.2.0)
#0 2.076   • Installing decorator (5.1.1)
#0 2.077   • Installing google-auth (2.16.2)
#0 2.078   • Installing googleapis-common-protos (1.58.0)
#0 2.078   • Installing greenlet (2.0.2)
#0 2.134   • Installing httplib2 (0.21.0)
#0 2.199   • Installing jedi (0.18.2)
#0 2.204   • Installing jupyter-core (5.3.0)
#0 2.206   • Installing marshmallow-enum (1.5.1)
#0 2.209   • Installing matplotlib-inline (0.1.6)
#0 2.328   • Installing pexpect (4.8.0)
#0 2.332   • Installing pickleshare (0.7.5)
#0 2.349   • Installing prompt-toolkit (3.0.38)
#0 2.388   • Installing pygments (2.14.0)
#0 2.417   • Installing python-dateutil (2.8.2)
#0 2.449   • Installing pyzmq (25.0.1)
#0 2.466   • Installing requests (2.28.2)
#0 2.492   • Installing stack-data (0.6.2)
#0 2.549   • Installing tornado (6.2)
#0 2.573   • Installing typing-inspect (0.8.0)
#0 2.651   • Installing yarl (1.8.2)
#0 3.267   • Installing aiohttp (3.8.4)
#0 3.267   • Installing click (8.1.3)
#0 3.268   • Installing comm (0.1.2)
#0 3.269   • Installing dataclasses-json (0.5.7)
#0 3.270   • Installing debugpy (1.6.6)
#0 3.271   • Installing google-auth-httplib2 (0.1.0)
#0 3.272   • Installing google-api-core (2.11.0)
#0 3.273   • Installing h11 (0.14.0)
#0 3.274   • Installing ipython (8.11.0)
#0 3.335   • Installing jupyter-client (8.0.3)
#0 3.345   • Installing nest-asyncio (1.5.6)
#0 3.350   • Installing numpy (1.24.2)
#0 3.413   • Installing pathspec (0.11.1)
#0 3.422   • Installing psutil (5.9.4)
#0 3.458   • Installing pydantic (1.10.6)
#0 3.470   • Installing pyyaml (6.0)
#0 3.494   • Updating setuptools (65.5.1 -> 67.6.0)
#0 3.564   • Installing soupsieve (2.4)
#0 3.710   • Installing sqlalchemy (1.4.46)
#0 3.806   • Installing starlette (0.24.0)
#0 3.885   • Installing tenacity (8.2.2)
#0 4.049   • Installing tomli (2.0.1)
#0 4.065   • Installing tqdm (4.65.0)
#0 4.079   • Installing uritemplate (4.1.1)
#0 4.946 
#0 4.946   _WheelFileValidationError
#0 4.946 
#0 4.946   ["In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_cython.cp39-win_amd64.pyd didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.cp39-win_amd64.pyd didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_amd64.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_x86.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_amd64.exe didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_x86.exe didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_amd64.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_x86.dll didn't match RECORD"]
#0 4.947 
#0 4.947   at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/installer/sources.py:289 in validate_record
#0 4.954       285│                         f"In {self._zipfile.filename}, hash / size of {item.filename} didn't match RECORD"
#0 4.954       286│                     )
#0 4.954       287│ 
#0 4.954       288│         if issues:
#0 4.954     → 289│             raise _WheelFileValidationError(issues)
#0 4.954       290│ 
#0 4.954       291│     def get_contents(self) -> Iterator[WheelContentElement]:
#0 4.954       292│         """Sequential access to all contents of the wheel (including dist-info files).
#0 4.954       293│ 
#0 4.954 
#0 5.440 
#0 5.440   ChefBuildError
#0 5.440 
#0 5.440   Backend subprocess exited when trying to invoke build_wheel
#0 5.440   
#0 5.440   running bdist_wheel
#0 5.440   running build
#0 5.440   running build_py
#0 5.440   creating build
#0 5.440   creating build/lib.linux-aarch64-cpython-310
#0 5.440   creating build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_pswindows.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_psbsd.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_pssunos.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/__init__.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_psaix.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_psosx.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_pslinux.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_common.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_psposix.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   copying psutil/_compat.py -> build/lib.linux-aarch64-cpython-310/psutil
#0 5.440   creating build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/runner.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_system.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/__init__.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_process.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   copying psutil/tests/__main__.py -> build/lib.linux-aarch64-cpython-310/psutil/tests
#0 5.440   running build_ext
#0 5.440   building 'psutil._psutil_linux' extension
#0 5.440   creating build/temp.linux-aarch64-cpython-310
#0 5.440   creating build/temp.linux-aarch64-cpython-310/psutil
#0 5.440   gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=594 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/tmp/tmprc02belu/.venv/include -I/usr/local/include/python3.10 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-cpython-310/psutil/_psutil_common.o
#0 5.440   C compiler or Python headers are not installed on this system. Try to run:
#0 5.440   sudo apt-get install gcc python3-dev
#0 5.440   error: command 'gcc' failed: No such file or directory
#0 5.440   
#0 5.440 
#0 5.440   at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chef.py:152 in _prepare
#0 5.445       148│ 
#0 5.445       149│                 error = ChefBuildError("\n\n".join(message_parts))
#0 5.445       150│ 
#0 5.445       151│             if error is not None:
#0 5.445     → 152│                 raise error from None
#0 5.445       153│ 
#0 5.445       154│             return path
#0 5.445       155│ 
#0 5.445       156│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
#0 5.445 
#0 5.445 Note: This error originates from the build backend, and is likely not a problem with poetry but with psutil (5.9.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "psutil (==5.9.4)"'.
#0 5.445 
------
failed to solve: executor failed running [/bin/sh -c poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi]: exit code: 1

[ogabrielluiz]

Hey! I'm running on a Mac too and I encountered this error.
I'll add gcc to the dev.Dockerfile and make a pull request ASAP.

I think that is all it takes so you can do it locally while the PR is accepted.

[ScripterSugar]

I'm running docker-compose up --build from mac, in the dev branch with your changes applied, and still getting the buildtime error on installation. fyi, I'm running command on the macbook pro with apple M1 chip, if that matters. full error logs are like following:

_WheelFileValidationError
#0 6.491 
#0 6.491   ["In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_cython.cp39-win_amd64.pyd didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.cp39-win_amd64.pyd didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_amd64.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_x86.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_amd64.exe didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_x86.exe didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_amd64.dll didn't match RECORD", "In /root/.cache/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_x86.dll didn't match RECORD"]
#0 6.491 
#0 6.491   at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/installer/sources.py:289 in validate_record
#0 6.502       285│                         f"In {self._zipfile.filename}, hash / size of {item.filename} didn't match RECORD"
#0 6.502       286│                     )
#0 6.502       287│ 
#0 6.502       288│         if issues:
#0 6.502     → 289│             raise _WheelFileValidationError(issues)
#0 6.502       290│ 
#0 6.502       291│     def get_contents(self) -> Iterator[WheelContentElement]:
#0 6.502       292│         """Sequential access to all contents of the wheel (including dist-info files).
#0 6.502       293│ 
#0 6.502 
------
failed to solve: executor failed running [/bin/sh -c poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi]: exit code: 1

any help?

[ScripterSugar]

Update: I think this issue is originated from debugpy 1.6.6 installation with poetry.

Here's the issue link in debugpy repo: microsoft/debugpy#1246
Maybe they did something wrong with the wheel configuration?

[ScripterSugar]

If anyone looking for a quick fix, setting poetry's installer config installer.modern-installation to false can temporarily fix my issue by running poetry config installer.modern-installation false either in buildtime from docker or in your local machine.

[ogabrielluiz]

In my machine it normally breaks when installing numcodecs. I haven't had problems with building for a few days now and I think the reasons this is working might that I activated Rosetta in Docker, or that I added gcc installation.

I'll run some more tests to see if I can pinpoint the solution.

Installing gcc did help with fixing that error in other environments besides M1 Mac.

[ionflow]

poetry config installer.modern-installation false

That works for me too! Thanks @ScripterSugar!