Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lz4 fails to cross-compile due to gen_manual rule #1240

Closed
riastradh opened this issue Jun 14, 2023 · 4 comments
Closed

lz4 fails to cross-compile due to gen_manual rule #1240

riastradh opened this issue Jun 14, 2023 · 4 comments

Comments

@riastradh
Copy link

Describe the bug
lz4 uses $(CXX) to compile gen_manual, and then executes it during the build.

In a cross-build, say using a cross-compiler on an x86 system to build an lz4 that runs on aarch64, $(CXX) is a cross-compiler, so using it here yields an aarch64 executable for gen_manual, which can't be executed on x86.

Expected behavior
lz4 uses a native compiler for gen_manual when cross-compiling everything else.

For example, perhaps lz4 could sprout a parameter CXX_FOR_BUILD, which defaults to $(CXX), and which is used to compile gen_manual before executing it.

To Reproduce
Cross-compile lz4.

Additional context
Many other software packages that do something similar expose CC_FOR_BUILD or CXX_FOR_BUILD parameters, so there is plenty of precedent for this approach.

riastradh pushed a commit to riastradh/lz4 that referenced this issue Jun 14, 2023
This way, in a cross-build of lz4, you can set CXX_FOR_BUILD to a
native compiler that can produce a gen_manual executable to run at
build-time.

fix lz4#1240
@Cyan4973
Copy link
Member

A number of things to unpack here.

  • gen_manual is not part of the standard lz4 package. If one runs the command make, this will not generate (nor run) gen_manual.
  • gen_manual is part of an extended list of binaries that can be created from the lz4 repository. This extended list also includes for example test programs, such as tests/fuzzer . The command make all can be used to generate this extended list. It's primarily useful for CI tests.
  • If your goal is to generate a binary package for a cross-compiled target, I'm not sure to understand why it would need the extended list of binaries, as opposed to the "core" useful package, featuring only the libraries and the CLI.
  • The goal of gen_manual is just to generate a manual in html format from the source code. It's a repository-level tool. There should not be any reason to run this binary on client side, the html manual is supposed to be already updated.

So, I suspect the first question is, we should talk about the need to generate gen_manual,
and if there is no such need,
why is it generated, can or should it be excluded ?

@riastradh
Copy link
Author

I came here from pkgsrc, which follows the convention that all is the standard make target for building everything important in a source tree. The pkgsrc packaging for lz4 doesn't do anything to override this.

I didn't investigate whether what gen_manual produces is relevant to the resulting package -- I'm just going through and fixing cross-compilation bugs in every package I stumble upon, and this is a common bug with a common fix, which is what I submitted in #1241.

Is there a better target to use instead? Should it use the default target?

@riastradh
Copy link
Author

Looks like there's no trouble with leaving the target empty in pkgsrc so it will pick whatever is the default from make, which avoids the cross-compilation issue. Thanks!

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jun 15, 2023
The `all' target apparently includes things that aren't necessary for
the build products we expect.  Just use an empty target so that make
selects the first one in the makefile, which according to
lz4/lz4#1240 is the reasonable choice here.

No change to the build products that are installed -- this just makes
lz4 build fewer things.
@Cyan4973
Copy link
Member

Cyan4973 commented Jun 15, 2023

Yeah, the regular default make is the one intended to build lz4 packages.

make all is essentially used in CI, to build everything in the repository, and ensure all targets, including secondary ones, still work fine after every modification.

Maybe it's a mis-usage of the all target. I should have a deeper look at that, understand what's the established convention and see if it's respected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants