Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Warning: unmatched variable LUA_LIBDIR #1155

Closed
doronbehar opened this issue Feb 12, 2020 · 10 comments
Closed

Warning: unmatched variable LUA_LIBDIR #1155

doronbehar opened this issue Feb 12, 2020 · 10 comments

Comments

@doronbehar
Copy link

On NixOS, since 3.3.1 was released, we are having an issue with a certain cmake build. The package we are failing to install is https://github.com/luvit/luv. Here's the issue open on our end: NixOS/nixpkgs#79870 However, if from luarocks' repo, in master I run luarocks --tree=test-luv install luv the build doesn't fail, yet the warning message:

Warning: unmatched variable LUA_LIBDIR

And also Cmake reports this variable as empty:

-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/9d6mlidaj99j673j583dgwlbj2b7yhnq-luajit-2.1.0-beta3/include
-- LUA: /nix/store/9d6mlidaj99j673j583dgwlbj2b7yhnq-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND

It's likely that we'll need to update something in our build infrastructure, so I'm trying to figure out what it is and the meaning of this warning. Help will be appreciated.
@doronbehar doronbehar mentioned this issue Feb 12, 2020
Closed
@hishamhm
Copy link
Member

since 3.3.1 was released

@doronbehar Last known working version is 3.2.1, right?

Are you able to test this fetching from a specific LuaRocks commit? Looking at the commits between 3.2.1 and 3.3.1, I suspect 83126ba is the culprit, so testing revision 9499bb8 could answer the question.

@doronbehar
Copy link
Author

Oh right, I've reached the conclusion that I'll have to test everything with our build system because using luarocks install is likely to act very different than our build system does.

I've tested the build of the failing package with 3.2.1 and the warning and even the error were there.
It seems the difference between 3.2.1 and 3.3.1 is that 3.3.1 exits with a non zero status and that tells our build system to abort because an error occurred. Oddly, that error / warning doesn't cause any of the packages that depend on this package to fail to build/run.

Here's the log of the build with 3.2.1 which isn't considered to fail:

building '/nix/store/24mrrcihp7abs3ry0rrsxpcc6kd7sc33-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/bin/lua"
installing

Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/gk304dpcw0x4fx7vp3di9v1lv9s8rhdc-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1.30.0
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory
No existing manifest. Attempting to rebuild...
luv 1.30.0-0 is now installed in /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0 (license: Apache 2.0)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0
shrinking /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so.1
shrinking /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so.1.30.0
strip is /nix/store/xpzdkp9y8fnliwrs2g5pygx2qmmbrv8w-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0/lib
patching script interpreter paths in /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0
checking for references to /build/ in /nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0...
/nix/store/iwh1advcncfdp6iqjhqrml1167l980i8-luajit-2.1.0-beta3-luv-1.30.0-0

As for 9499bb8, I've tested it and the build failed the same way it failed with 3.3.1:

building '/nix/store/rj9manifwqzl68r0zmfrgx0lg3aw479p-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.30.0-0 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is GNU 9.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc
-- Check for working C compiler: /nix/store/fqhjxf9ii4w4gqcsx59fyw2vvj91486a-gcc-wrapper-9.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/gk304dpcw0x4fx7vp3di9v1lv9s8rhdc-libuv-1.34.2/lib/libuv.so
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/jq08mbd6f4b67r4d7frm7swv9kc29j3y-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /build/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.so
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1.30.0
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so.1
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory

Error: Failed copying /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so to /nix/store/ggi0zjjlsqkzyxia3gshbx9m4girimr3-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so
builder for '/nix/store/rj9manifwqzl68r0zmfrgx0lg3aw479p-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed with exit code 1
error: build of '/nix/store/rj9manifwqzl68r0zmfrgx0lg3aw479p-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed

@doronbehar doronbehar mentioned this issue Feb 13, 2020
Closed
10 tasks
@hishamhm
Copy link
Member

hishamhm commented Feb 13, 2020 •
edited

@doronbehar it's interesting that the 3.2.1 logs already show an empty LUA_LIBDIR. Do you have the full command-line of the luarocks command that fails, and its output logs? (on both 3.2.1 where the error is not fatal, and on 3.3.1)

@hishamhm
Copy link
Member

It may also be a good idea to try to run luarocks with --verbose (which should give us a ton more logs).

@doronbehar
Copy link
Author

doronbehar commented Feb 13, 2020 •
edited

While digging into our build system, I've discovered we use an automatically generated config file for every build of a Luarocks package. It's content are here:

Unfortunately, I couldn't find a way to get it's contents for the build of libluv - with the Nix lang variables expanded.

However, as for running the build with luarocks --verbose, that was easy so here are the full logs for each build of Luarocks:

It's likely that you'll notice trace messages from other programs due to the fact I used a general Nix debug flag - it causes our compiler wrappers for example to be more verbose.

@hishamhm
Copy link
Member

I see similar errors on both 3.2.1:

cp: cannot stat '/nix/store/1w7vlgh1srach0vg1rgl5bac07m86gyl-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory

and 3.3.1:

Error: Failed copying /nix/store/hjk2mi8r4ggfjl7s8g442s3hirlcd8z7-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so to
                      /nix/store/hjk2mi8r4ggfjl7s8g442s3hirlcd8z7-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so

Looks like it was ignored in 3.2.1 but caught in 3.3.1... I even suspected if it was in issue in the luv rockspec, but I just tried it locally and luarocks install luv built it fine... I'll dig some more, but at this point I guess we'll have to bisect between 3.2.1 and 3.3.1 on your end to find the culprit...

@doronbehar
Copy link
Author

I've traced down the issue to commit 3a3f2cb - building luv with luarocks built from 879c6ef which is right before 3a3f2cb doesn't fail.

Here's the not failing build log with 879c6ef : https://gist.github.com/856e78c672a2b0b8385b01116f62f77a

And the failing build with commit 3a3f2cb : https://gist.github.com/406902127e91cf02a582f4906205d4b3

@hishamhm
Copy link
Member

@doronbehar Super interesting! Thank you for digging down to the specific commit!

What's most interesting about this is that the central thing that this commit does is to add error checking:

3a3f2cb#diff-09933fcfab73d4a2d7c8ea3a9dad2ac7R446-R460

This means that an error was already happening before, but it was just not being caught and the process succeeded. The not-failing log in 879c6ef does include this:

os.execute:     cd '/build/luv-1.30.0-0/luv-1.30.0-0' && cp '/nix/store/ppjcblk2dvgj4lq9r7gdzparzrvprwfw-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so' '/nix/store/ppjcblk2dvgj4lq9r7gdzparzrvprwfw-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.so'
cp: cannot stat '/nix/store/ppjcblk2dvgj4lq9r7gdzparzrvprwfw-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.so': No such file or directory
Results: 1
  1 (number): 256

I'll keep looking into this!

@doronbehar
Copy link
Author

Oh please I don't think you should dig too much into this. Apparently @ArdaXi has just confirmed that if we update luv to the latest version of it, the build passes with the latest Luarocks. I wonder though what it was that they changed between 1.30 and 1.34..

@hishamhm
Copy link
Member

Oh nice! This means that the new LuaRocks is catching the error correctly (whereas the previous one didn't) and whatever was causing the error was fixed in the latest version of the dependency. Mystery solved, then, happy to close this one! Thank you for all the debugging and detailed feedback!

@doronbehar doronbehar mentioned this issue Feb 27, 2020
Closed
10 tasks
@doronbehar doronbehar mentioned this issue Jun 8, 2020
Merged
10 tasks
@doronbehar doronbehar mentioned this issue Aug 14, 2020
Merged
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hishamhm @doronbehar