luaposix does not cross compile properly

[joakim-tjernlund]

Consider this log snipped:

arm-xrmllib32-linux-musleabi-gcc  -march=armv8-a+crc -mcpu=cortex-a53 -fstack-protector-strong  -Os -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/builduser/workspace/XR/XR_PretestJobs/yocto_pipeline/build/build/infn-xr/gmcu/tmp/work/cortexa53-32-xrmllib32-linux-musleabi/lib32-luaposix/35.1-r0/lib32-recipe-sysroot -O2 -shared -fPIC   -DPACKAGE='"luaposix"' -D_XOPEN_SOURCE=700 -D_POSIX_C_SOURCE=200809L -D_FORTIFY_SOURCE=2 -DNDEBUG -DVERSION='"builduser"' -Iext/include -I/usr/include/lua5.4 ext/posix/signal.c -o linux/posix/signal.so -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fmacro-prefix-map=/home/builduser/workspace/XR/XR_PretestJobs/yocto_pipeline/build/build/infn-xr/gmcu/tmp/work/cortexa53-32-xrmllib32-linux-musleabi/lib32-luaposix/35.1-r0=/usr/src/debug/lib32-luaposix/35.1-r0                      -fdebug-prefix-map=/home/builduser/workspace/XR/XR_PretestJobs/yocto_pipeline/build/build/infn-xr/gmcu/tmp/work/cortexa53-32-xrmllib32-linux-musleabi/lib32-luaposix/35.1-r0=/usr/src/debug/lib32-luaposix/35.1-r0                      -fdebug-prefix-map=/home/builduser/workspace/XR/XR_PretestJobs/yocto_pipeline/build/build/infn-xr/gmcu/tmp/work/cortexa53-32-xrmllib32-linux-musleabi/lib32-luaposix/35.1-r0/lib32-recipe-sysroot=                      -fdebug-prefix-map=/home/builduser/workspace/XR/XR_PretestJobs/yocto_pipeline/build/build/infn-xr/gmcu/tmp/work/cortexa53-32-xrmllib32-linux-musleabi/lib32-luaposix/35.1-r0/recipe-sysroot-native=  -Wl,-z,relro,-z,now
[2023-09-07T14:27:47.651Z] | cc1: warning: include location "/usr/include/lua5.4" is unsafe for cross-compilation [-Wpoison-system-directories]
[2023-09-07T14:27:47.651Z] | In file included from /usr/include/lua5.4/lua.h:16,
[2023-09-07T14:27:47.651Z] |                  from ext/include/_helpers.c:59,
[2023-09-07T14:27:47.651Z] |                  from ext/posix/signal.c:25:
[2023-09-07T14:27:47.651Z] | /usr/include/lua5.4/luaconf.h:43:9: error: #error "No ABI matched

hardcodes -I/usr/include/lua5.4
This comes from luke line:

luke.substitute=merge(luke.substitute or{},{package=interpolate_to_substitute(luke.package),version=interpolate_to_substitute(luke.version),})luke.variables=merge(luke.variables or{},collect_variables(luke),{LUA_DIR='/usr',LUA_BINDIR='$LUA_DIR/bin',LUA_INCDIR='$LUA_DIR/include/lua$LUAVERSION',LUA_LIBDIR='$LUA_DIR/lib',objdir=platforms[1],package=luke.package,version=luke.version,})return{clidefs=parsed.clidefs,install=install,log=parsed.log,luke=luke,valreqs=parsed.valreqs,verbose=parsed.verbose,write=parsed.write,}end,}

I guess inc dir can be just LUA_INCDIR=$LUAVERSION ?
Not sure what to with BINDIR/LIBDIR though? Just remove them?

[gvvaughan]

I don't have any cross compilation environment or way to develop/test code to support it.
But I'd be very happy to merge PRs that allow you to easily run a cross compiled build. While autoconf was used here several years ago, I know cross compilation worked put of the box, but the amount of code dropped by autotools was an order of magnitude larger than luaposix itself, and luke was my reaction to that by shipping a small understandable self contained tool for cross platform build support.

Note that the build-aux/luke script in this repo is minified - you'll find the complete sources at https://github.com/gvvaughan/luke which are much much easier to follow.

To answer your questions: LUA_INCDIR should work as is, unless you are building for a different version of Lua. LUA_LIBDIR should point to the directory for the cross-compiled Lua libraries libposix should be linked against. LUA_BINDIR is where Lua binaries that will run on the host machine are.

[joakim-tjernlund]

I did the quickest fix I could think of add a = in here: LUA_INCDIR='=$LUA_DIR/
That make gcc inject sysroot path before /usr/include
Here is a crude patch:

--- ./build-aux/luke.org	2023-09-12 16:08:47.195972896 +0200
+++ ./build-aux/luke	2023-09-12 16:09:16.932990694 +0200
@@ -79,7 +79,7 @@
 if isempty(luke.modules)then
 luke.external_dependencies=nil
 end
-luke.substitute=merge(luke.substitute or{},{package=interpolate_to_substitute(luke.package),version=interpolate_to_substitute(luke.version),})
luke.variables=merge(luke.variables or{},collect_variables(luke),{LUA_DIR='/usr',LUA_BINDIR='$LUA_DIR/bin',LUA_INCDIR='$LUA_DIR/include/lua$LUA
VERSION',LUA_LIBDIR='$LUA_DIR/lib',objdir=platforms[1],package=luke.package,version=luke.version,})return{clidefs=parsed.clidefs,install=instal
l,log=parsed.log,luke=luke,valreqs=parsed.valreqs,verbose=parsed.verbose,write=parsed.write,}end,}
+luke.substitute=merge(luke.substitute or{},{package=interpolate_to_substitute(luke.package),version=interpolate_to_substitute(luke.version),})
luke.variables=merge(luke.variables or{},collect_variables(luke),{LUA_DIR='/usr',LUA_BINDIR='$LUA_DIR/bin',LUA_INCDIR='=$LUA_DIR/include/lua$LU
AVERSION',LUA_LIBDIR='$LUA_DIR/lib',objdir=platforms[1],package=luke.package,version=luke.version,})return{clidefs=parsed.clidefs,install=insta
ll,log=parsed.log,luke=luke,valreqs=parsed.valreqs,verbose=parsed.verbose,write=parsed.write,}end,}
 end
 package.preload['luke.compile']=function()
 local _ENV=require'std.normalize'{'luke._base','luke.environment','std.functional','type.context-manager','type.path',SHELLMETACHARS='[%s%$"]'
,}local function spawn(env,...)local command=interpolate(env,concat({...},' '))return with(TmpFile(),TmpFile(),function(out,err)local pipe=conc
at{command,' >',out.filename,' 2>',err.filename,'; printf $?'}return tonumber(slurp(Pipe(pipe))),slurp(File(err.filename)),slurp(File(out.filen
ame))end)end

[joakim-tjernlund]

The right fix would be to use pkgconf:

# > pkgconf lua5.4 --libs
-llua5.4 

 # > pkgconf lua5.4 --cflags-only-I
-I/usr/include/lua5.4 

[gvvaughan]

would it be sufficient to add a section to the README for cross compilation recommending building with

LUA_INCDIR="$(pkgconf lua$LUAVER --cflags-only-I)" LUA_LIBDIR="$(pkgconf lua$LUAVER --libs)"

Or would it be better to get that into the rockspec based on some condition set for luarocks?

[joakim-tjernlund]

would it be sufficient to add a section to the README for cross compilation recommending building with

I don't think so, not patching should be required.

LUA_INCDIR="$(pkgconf lua$LUAVER --cflags-only-I)" LUA_LIBDIR="$(pkgconf lua$LUAVER --libs)"

Or would it be better to get that into the rockspec based on some condition set for luarocks?

pkgconf/pkg-config should already be available on any system doing builds as many pkgs use it.

Also, some cross systems replaces pkg-config with a special cross version which is why most/all
SW pkgs uses $PKG_CONFIG in their builds.

You might need to do:

   PKG_CONFIG?=pkg-config

and then have:

$PKG_CONFIG --cflags-only-I

in luke script