Handle nullptr values in lualand

[karenzshea]

Hello again!

Came across some interesting behavior around null pointer handling. I made an example at https://github.com/karenzshea/sunless that can be compiled with ./build.sh null.cc from the project directory.

It compiles, but running the resulting binary results in a segfault à la

Program received signal SIGSEGV, Segmentation fault.
0x000000000040228c in std::char_traits<char>::length(char const*) ()
(gdb) bt
#0  0x000000000040228c in std::char_traits<char>::length(char const*) ()
#1  0x00000000004028df in sol::stack::pusher<char const*, void>::push(lua_State*, char const*) ()
#2  0x0000000000404eb8 in int sol::stack::push_reference<char const*>(lua_State*, char const*&&) ()
#3  0x0000000000404dc3 in int sol::stack::call_into_lua<false, char const*, , int, sol::wrapper<char const* (*)(int), void>::caller, char const* (*&)(int), void>(sol::types<char const*>, sol::types<int>, lua_State*, int, sol::wrapper<char const* (*)(int), void>::caller&&, char const* (*&)(int)) ()
#4  0x0000000000404a21 in int sol::call_detail::agnostic_lua_call_wrapper<char const* (*)(int), true, false, false, 0, void>::call<char const* (*&)(int)>(lua_State*, char const* (*&)(int)) ()
#5  0x000000000040499f in int sol::call_detail::call_wrapped<void, true, false, 0, char const* (*&)(int)>(lua_State*, char const* (*&)(int)) ()
#6  0x0000000000404507 in sol::function_detail::upvalue_free_function<char const* (&)(int)>::real_call(lua_State*) ()
#7  0x0000000000404536 in int sol::detail::static_trampoline<&sol::function_detail::upvalue_free_function<char const* (&)(int)>::real_call>(lua_State*) ()
#8  0x0000000000404391 in sol::function_detail::upvalue_free_function<char const* (&)(int)>::call(lua_State*) ()
#9  0x00007ffff7bb08ed in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#10 0x00007ffff7bbc4ed in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#11 0x00007ffff7bb0c2e in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#12 0x00007ffff7bb026f in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#13 0x00007ffff7bb0e71 in ?? () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#14 0x00007ffff7bacd91 in lua_pcallk () from /usr/lib/x86_64-linux-gnu/liblua5.2.so.0
#15 0x0000000000402950 in sol::stack::script(lua_State*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#16 0x0000000000402b0f in sol::state_view::script(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#17 0x0000000000401a8e in main ()

I can get around a crash by returning an empty string "" rather than nullptr, but lua scripts may be expecting a nil value rather than "".

[ThePhD]

Well. There's a number of things you (and I) can do to fix this:

1. (Speed) Let it continue to explode if this happens, and make the user specify sol::optional<const char*> as a return. Has the benefit of letting the user pick nil (sol::none) purposefully, but is API-intrusive
2. (Ease of use? A) Check for null, serialize an empty string if the const char* is null
3. (Ease of use? B) Check for null, seralize an Lua nil if const char* is null

I want to do 2, but 3 is just as valid, honestly, and probably better. 1 is the Most Holy Way™ but probably not worth it.

[ThePhD]

I made a decision: Picked 3. Makes the most sense in the end, I guess.

Get the latest: https://github.com/ThePhD/sol2/blob/develop/single/sol/sol.hpp

[karenzshea]

🙇‍♀️ Thanks much for the fix. Luabind's behavior was also to serialize lua's nil given null pointers, so hopefully this helps anyone else switching over.