DuckDB Build Failure in Alpine Linux Docker Image (host: mac M1)

[baxiry]

What happens?

While building a Image container alpine@latest
Pip did not find the DuckDB library with the appropriate version, so pip started building it from source, but then the build failed.

Host system: Mac M1.

To Reproduce

steps:

git clone https://github.com/edraj/dmart.git && cd dmart/admin_scripts/docker
# either
podman build -t dmart -f Dockerfile
# or
docker build . -t dmart -f Dockerfile

output:

gcc: fatal error: Killed signal terminated program cc1plus
      compilation terminated.
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -DDUCKDB_PYTHON_LIB_NAME=duckdb -DDUCKDB_EXTENSION_PARQUET_LINKED -DDUCKDB_EXTENSION_ICU_LINKED -DDUCKDB_EXTENSION_FTS_LINKED -DDUCKDB_EXTENSION_TPCH_LINKED -DDUCKDB_EXTENSION_TPCDS_LINKED -DDUCKDB_EXTENSION_JSON_LINKED -DDUCKDB_EXTENSION_AUTOLOAD_DEFAULT=1 -DDUCKDB_EXTENSION_AUTOINSTALL_DEFAULT=1 -Iduckdb_build/src/include -Iduckdb_build/third_party/concurrentqueue -Iduckdb_build/third_party/fast_float -Iduckdb_build/third_party/fastpforlib -Iduckdb_build/third_party/fmt/include -Iduckdb_build/third_party/fsst -Iduckdb_build/third_party/httplib -Iduckdb_build/third_party/hyperloglog -Iduckdb_build/third_party/jaro_winkler -Iduckdb_build/third_party/jaro_winkler/details -Iduckdb_build/third_party/libpg_query -Iduckdb_build/third_party/libpg_query/include -Iduckdb_build/third_party/lz4 -Iduckdb_build/third_party/mbedtls -Iduckdb_build/third_party/mbedtls/include -Iduckdb_build/third_party/mbedtls/library -Iduckdb_build/third_party/miniz -Iduckdb_build/third_party/pcg -Iduckdb_build/third_party/re2 -Iduckdb_build/third_party/skiplist -Iduckdb_build/third_party/tdigest -Iduckdb_build/third_party/utf8proc -Iduckdb_build/third_party/utf8proc/include -Iduckdb_build/third_party/yyjson/include -Iduckdb_build/extension/parquet/include -Iduckdb_build/third_party/parquet -Iduckdb_build/third_party/thrift -Iduckdb_build/third_party/lz4 -Iduckdb_build/third_party/snappy -Iduckdb_build/third_party/zstd/include -Iduckdb_build/third_party/mbedtls -Iduckdb_build/third_party/mbedtls/include -Iduckdb_build/extension/icu/include -Iduckdb_build/extension/icu/third_party/icu/common -Iduckdb_build/extension/icu/third_party/icu/i18n -Iduckdb_build/extension/fts/include -Iduckdb_build/third_party/snowball/libstemmer -Iduckdb_build/third_party/snowball/runtime -Iduckdb_build/third_party/snowball/src_c -Iduckdb_build/extension/tpch/include -Iduckdb_build/extension/tpch/dbgen/include -Iduckdb_build/extension/tpcds/include -Iduckdb_build/extension/tpcds/dsdgen/include -Iduckdb_build/extension/tpcds/dsdgen/include/dsdgen-c -Iduckdb_build/extension/json/include -Iduckdb_build -I/tmp/pip-install-_5zr5oez/duckdb_225311e590674cc6b7a8610c08ec87c0/src/include -I/tmp/pip-build-env-gy0e1xes/overlay/lib/python3.12/site-packages/pybind11/include -I/tmp/pip-build-env-gy0e1xes/overlay/lib/python3.12/site-packages/pybind11/include -I/home/venv/include -I/usr/include/python3.12 -c duckdb_build/extension/icu/./icu-makedate.cpp -o build/temp.linux-aarch64-cpython-312/duckdb_build/extension/icu/./icu-makedate.o -std=c++11 -g0
      error: command '/usr/bin/gcc' failed with exit code 1
      gcc: fatal error: Killed signal terminated program cc1plus
      compilation terminated.
      [end of output]

OS:

mac os m1

DuckDB Version:

i think latest

DuckDB Client:

python

Full Name:

baxiry

Affiliation:

dmart

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have

[baxiry]

https://pypi.org/project/duckdb/#files

alpine linux distribution uses musl lib
I think if there was a pre-compiled duckdb musl linux aarch64 then problem of building from source could be avoided.

[baxiry]

Update

Successfully building duckdb alpine container on Mac M1
I have updated podman version to 5.0.2
I also changed podman settings to 8gb of ram (my device doesn't really have that number)
Building duckdb took about 40 minutes.
Everything works fine
But I still think that providing a pre-build of musl linux ARM64 architecture is very important.

[sptrakesh]

The crash usually indicates the builder running out of memory. Did you try specifying higher memory limit (eg. -m 8192m) in docker build?

[baxiry]

Hi @sptrakesh
Yes, you are right.
The crash output did not indicate anything related to running out of memory. But I read somewhere that one of reasons for the crash may be due to insufficient memory. Exactly as you mentioned.
I changed memory settings in podman to 8GB and duckdb build succeeded.

I think the title of this issue should be changed

[kefahi]

On all other linux flavors (manylinux+arm/x86 and musl/x86) pip install duckdb simply installs the pre-compiled binary. while there is no pre-compiled binary for musl+arm.

To check the list of available pre-compiled binaries ... https://pypi.org/project/duckdb/#files

It would be great if musl + arm is pre-compiled as well for better Alpine on arm pip install duckdb experience.