Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Fix aarch64 cross build #3353

Merged
merged 1 commit into from Dec 10, 2021
Merged

Conversation

Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Dec 10, 2021

Signed-off-by: Xuanwo github@xuanwo.io

I hereby agree to the terms of the CLA available at: https://databend.rs/policies/cla/

Summary

This PR fixed aarch64 cross-build support.

And this PR will remove not well maintained target support including:

  • armv7-unknown-linux-gnueabihf
  • arm-unknown-linux-gnueabi

No one really uses them and they are not covered by CI.

Let's add them back until we are ready to maintain them.

NOTICE: After this PR get merged, we need to push the built image to dockerhub.

Detailed explain

The original rustembedded/cross:aarch64-unknown-linux-gnu image doesn't have lib we need, so cross build --target aarch64-unknown-linux-gnu will meet errros like:

  running: "aarch64-linux-gnu-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-I" "/usr/include" "-Wall" "-Wextra" "-E" "build/expando.c"
  cargo:warning=build/expando.c:2:33: fatal error: openssl/opensslconf.h: No such file or directory
  cargo:warning=compilation terminated.
  exit status: 1

  --- stderr
  thread 'main' panicked at '
  Header expansion error:
  Error { kind: ToolExecError, message: "Command \"aarch64-linux-gnu-gcc\" \"-O0\" \"-ffunction-sections\" \"-fdata-sections\" \"-fPIC\" \"-g\" \"-fno-omit-frame-pointer\" \"-I\" \"/usr/include\" \"-Wall\" \"-Wextra\" \"-E\" \"build/expando.c\" with args \"aarch64-linux-gnu-gcc\" did not execute successfully (status code exit status: 1)." }

  Failed to find OpenSSL development headers.

  You can try fixing this setting the `OPENSSL_DIR` environment variable
  pointing to your OpenSSL installation or installing OpenSSL headers package
  specific to your distribution:

      # On Ubuntu
      sudo apt-get install libssl-dev
      # On Arch Linux
      sudo pacman -S openssl
      # On Fedora
      sudo dnf install openssl-devel

  See rust-openssl README for more information:

      https://github.com/sfackler/rust-openssl#linux
  ', /cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.71/build/main.rs:162:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

In the past, we fixed this problem via self-compiled openssol with hardcoded OPENSSL_DIR and other envs.

RUN apt-get update && \
    apt-get install -y wget
COPY docker/build-tool/openssl.sh /
RUN bash /openssl.sh linux-armv4 arm-linux-gnueabihf-
ENV OPENSSL_DIR=/openssl \
    OPENSSL_INCLUDE_DIR=/openssl/include \
    OPENSSL_LIB_DIR=/openssl/lib

However, it's not enough. We will meet other problems like the following:

  = note: /usr/bin/ld: cannot find -lz

error: build failed

The real problem here is we pkg-config doesn't have current path.

So we only need to set the PKG_CONFIG_PATH correctly like the following:

ENV PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig:$PKG_CONFIG_PATH

And we can add new packages like:

RUN dpkg --add-architecture arm64 && \
    apt-get update && \
    apt-get install --assume-yes libssl-dev zlib1g-dev

The simple, perfect solution.

Changelog

  • Build/Testing/CI

Signed-off-by: Xuanwo <github@xuanwo.io>
@databend-bot databend-bot added the pr-build this PR changes build/testing/ci steps label Dec 10, 2021
@databend-bot
Copy link
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

1 similar comment
@databend-bot
Copy link
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@vercel
Copy link

vercel bot commented Dec 10, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/3U3pYUteRGV5mADJ5Qiqq2Pn6J6W
✅ Preview: Canceled

@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 10, 2021

This PR will fix build error in #3344.

@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 10, 2021

PTAL @PsiACE @ZhiHanZ

Copy link
Member

@PsiACE PsiACE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@databend-bot
Copy link
Member

Wait for another reviewer approval

@BohuTANG BohuTANG merged commit 1bdeb34 into datafuselabs:main Dec 10, 2021
@Xuanwo Xuanwo deleted the fix-corss-build branch December 10, 2021 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need-review pr-build this PR changes build/testing/ci steps
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants