Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llvm-config usage in cross compilation? #58984

Open
kanavin opened this issue Nov 14, 2022 · 8 comments
Open

llvm-config usage in cross compilation? #58984

kanavin opened this issue Nov 14, 2022 · 8 comments
Labels
llvm-tools All llvm tools that do not have corresponding tag

Comments

@kanavin
Copy link

kanavin commented Nov 14, 2022

When cross compiling items such as mesa with llvm support, the build process relies on output of llvm-config, and therefore on ability to run llvm-config. But if llvm as a whole has been cross-built, then it is not possible to run llvm-config because it needs a different computer.

How should this situation be handled? One option is to make llvm-config a arch-independent script (it is now a cpp program). Another is to also make it a script, but have it be only a thin wrapper over llvm's pkg-config file (which currently does not exist).

Suggestions?

Yocto project has until now dealt with this by

  • building llvm-config first for the build machine, then running that. This does not work so well due to mismatches between host and target configurations.
  • providing own replacement for llvm-config as a shell script. This also isn't great as its output is not kept in sync with what actual llvm-config produces.
@EugeneZelenko EugeneZelenko added llvm-tools All llvm tools that do not have corresponding tag and removed new issue labels Nov 14, 2022
@tstellar
Copy link
Collaborator

Internally, within the LLVM sub-projects, we have been migrating away from llvm-config and instead relying on the installed cmake files to provide the necessary information. Would you be able to do the same?

@kanavin
Copy link
Author

kanavin commented Nov 14, 2022

Thanks - the issue originates in meson build system (that mesa is using), and so I filed a ticket with them:
mesonbuild/meson#11043

kraj pushed a commit to YoeDistro/poky that referenced this issue Nov 14, 2022
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

(From OE-Core rev: 9e18167bdef3bf3ab7f4ef0d926061b0869643cd)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
@eli-schwartz
Copy link

Internally, within the LLVM sub-projects, we have been migrating away from llvm-config and instead relying on the installed cmake files to provide the necessary information. Would you be able to do the same?

Does this mean that LLVM has officially dropped support for being used in projects that choose a build system other than cmake?

If not, then how should projects that use a build system other than cmake get information out of a cmake program?

@tstellar
Copy link
Collaborator

Internally, within the LLVM sub-projects, we have been migrating away from llvm-config and instead relying on the installed cmake files to provide the necessary information. Would you be able to do the same?

Does this mean that LLVM has officially dropped support for being used in projects that choose a build system other than cmake?

No, but for something like cross-compiling, using the CMake files is going to be much easier than trying to build a working cross-compiled llvm-config.

@dcbaker
Copy link

dcbaker commented Nov 15, 2022

Would you take patches to add pkg-config files? That would probably solve the issue entirely for everyone who's not using cmake, maybe even removing the need for llvm-config altogether.

@tstellar
Copy link
Collaborator

Would you take patches to add pkg-config files? That would probably solve the issue entirely for everyone who's not using cmake, maybe even removing the need for llvm-config altogether.

I'm not sure how others feel, but I am in favor of this.

rpurdie pushed a commit to yoctoproject/poky that referenced this issue Nov 15, 2022
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

(From OE-Core rev: 056431883e94296b767a479d029b914392e4fd7c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Nov 15, 2022
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
@pogo59
Copy link
Collaborator

pogo59 commented Nov 15, 2022

Seems like tblgen would be in the same boat--it needs to run on the build host, not the target. Do we get that right? If so, could llvm-config be handled the same way? If not, that's its own issue.

@kanavin
Copy link
Author

kanavin commented Nov 15, 2022

I believe we build llvm-tblgen separately as a native binary, then install it somewhere where llvm cross build can use it:

                  -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \

llvm-config can't be handled this way, since you do need the cross version with cross settings in it.

halstead pushed a commit to openembedded/openembedded-core that referenced this issue Dec 1, 2022
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 0564318)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Dec 1, 2022
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

(From OE-Core rev: d2860e67a37623b17fc9c6e1192f06a63b7d52ff)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 056431883e94296b767a479d029b914392e4fd7c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
foxsen pushed a commit to foxsen/poky that referenced this issue Jan 2, 2023
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

(From OE-Core rev: 056431883e94296b767a479d029b914392e4fd7c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
daregit pushed a commit to daregit/yocto-combined that referenced this issue May 22, 2024
Sadly the magic is not perfect: llvm-config contains a hardcoded
value for the libdir from the native configuration, and things will
break if the target build installs libraries somewhere else (e.g. lib64).

llvm-config in target bindir also needs a rpath adjustment,
otherwise it simply won't even run when e.g. building for arm on x86.

To avoid patching llvm-source let's simply add more cases to the
llvm-config wrapper script, so that falling through to llvm-config binary
is avoided. Fortunately those cases are all static in what they return,
even though llvm-config binary does poke around the file tree to arrive
at them (which is where breakage happens if native and target don't match
exactly wrt libdir).

I verified that this works by building mesa with llvm enabled for
qemuarm64 and with baselib set to 'lib64' - so that both the target
architecture and target libdir differ from native ones.

Upstream tickets:
llvm/llvm-project#58984
mesonbuild/meson#11043

(From OE-Core rev: 056431883e94296b767a479d029b914392e4fd7c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
llvm-tools All llvm tools that do not have corresponding tag
Projects
None yet
Development

No branches or pull requests

6 participants