Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test dockerized compilation in Alpine:latest and Ubuntu:20.04 #11708

Merged
merged 3 commits into from
Apr 18, 2024

Conversation

carlopi
Copy link
Contributor

@carlopi carlopi commented Apr 18, 2024

Adding CI workflow for testing DuckDB compilation in dockerized environments.

Script currently provided for Alpine:latest and Ubunutu:20.04.
Those double down also as blueprints for how to compile and build DuckDB main binaries in those architectures / what dependencies are needed.

More tests are very welcome, it's a matter of adding a line per distribution to scripts/test_docker_images.sh.

Adding a partial revert of #11663 where I propose a different fix to #11640 via providing a define, unsure whether this will be sufficient. Asking @zmbc if he could either provide a docker script showing the problem, I haven't managed to reproduce, and/or testing in its configuration with this alternative solution.

In the current form fixes #11703, otherwise merging only the first commit this should at the very least provide a reproduction and means to avoid similar regressions.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 18, 2024 09:30
@carlopi carlopi marked this pull request as ready for review April 18, 2024 09:30
@carlopi
Copy link
Contributor Author

carlopi commented Apr 18, 2024

Push for these tests is by @szarnyasg, adding in as reviewer.

@carlopi carlopi requested a review from szarnyasg April 18, 2024 09:34
@carlopi carlopi marked this pull request as draft April 18, 2024 09:37
@carlopi carlopi marked this pull request as ready for review April 18, 2024 09:38
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 18, 2024 11:14
@carlopi carlopi marked this pull request as ready for review April 18, 2024 11:15
@carlopi
Copy link
Contributor Author

carlopi commented Apr 18, 2024

Also pinging @Flogex, since a PR of theirs (#11635) introduced the CMAKE_CURRENT_FUNCTION_LIST_DIR that is malfunctioning in the docker setup.

I have yet to check on their setup, at Flogex/duckdb-extension-template@7aa87de, to double check whether this change might work for everyone.

@Flogex
Copy link
Contributor

Flogex commented Apr 18, 2024

@carlopi Looks like CMAKE_CURRENT_FUNCTION_LIST_DIR is new in CMake 3.17, the minimum required version for DuckDB is 3.5. I didn't check this, sorry.

Using DUCKDB_MODULE_BASE_DIR as introduced in this PR works in a setup that includes the DuckDB CMake project as subdirectory and is the nicer approach IMO. 👍

@Mytherin Mytherin merged commit 1ffcc8b into duckdb:main Apr 18, 2024
50 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

@zmbc
Copy link
Contributor

zmbc commented Apr 19, 2024

Unfortunately, this fix doesn't work in a conda environment, in the same way as described in #11640. It is enough to call fallocate but does not include the flags. I am a bit out of my depth as to why it seems that in many setups, the falloc headers are included automatically, apparently including Alpine, although a linux-headers package is available for Alpine. Any pointers?

@carlopi
Copy link
Contributor Author

carlopi commented Apr 19, 2024

Unfortunately, this fix doesn't work in a conda environment, in the same way as described in #11640. It is enough to call fallocate but does not include the flags. I am a bit out of my depth as to why it seems that in many setups, the falloc headers are included automatically, apparently including Alpine, although a linux-headers package is available for Alpine. Any pointers?

I am happy to dig deeper, would it be possible for you to share instructions on how to reproduce your setup? Basically something in the form like:

docker run -i --rm -v $(pwd):/duckdb --workdir /duckdb ubuntu:20.04 <<< "apt-get update && export DEBIAN_FRONTEND=noninteractive && apt-get install g++ git make cmake ninja-build -y && GEN=ninja make && make clean" 2>&1

where instead you provide how to install conda + how to fetch dependencies + what exact packages are needed ?
I am happy to dig further, but I think you are in a better place to provide description of the exact steps you are following / your setup.

Once I can reproduce, it should be easy(er) to find a way out plus have a way to test no regression will happen for relevant setups.

github-actions bot pushed a commit to duckdb/duckdb-r that referenced this pull request May 1, 2024
Merge pull request duckdb/duckdb#11708 from carlopi/docker_builds
Merge pull request duckdb/duckdb#11713 from carlopi/no_python_builds
Merge pull request duckdb/duckdb#11701 from Mytherin/improvestatementreduction
Merge pull request duckdb/duckdb#11700 from Mytherin/bumpjulia
@carlopi carlopi deleted the docker_builds branch May 7, 2024 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Builds on Alpine linux are failing
4 participants