Add support for full-range 64 bit lightuserdata.

(cherry picked from commit e9af1ab)

LuaJIT uses special NaN-tagging technique to store internal type on
the Lua stack. In case of LJ_GC64 the first 13 bits are set in special
NaN type (0xfff8...). The next 4 bits are used for an internal LuaJIT
type of object on stack. The next 47 bits are used for storing this
object's content. For userdata, it is its address. For arm64 a pointer
can have more than 47 significant bits [1]. In this case the error BADLU
error is raised.

For the support of full 64-bit range lightuserdata pointers two new
fields in GCState are added:

`lightudseg` - vector of segments of lightuserdata. Each element keeps
32-bit value. 25 MSB equal to MSB of lightuserdata 64-bit address, the
rest are filled with zeros. The length of the vector is power of 2.

`lightudnum` - the length - 1 of aforementioned vector (up to 255).

When lightuserdata is pushed on the stack, if its segment is not stored
in vector new value is appended to of this vector. The maximum amount of
segments is 256. BADLU error is raised in case when user tries to add
userdata with the new 257-th segment, so the whole VA-space isn't
covered by this patch.

Also, in this patch all internal usage of lightuserdata (for hooks,
profilers, built-in package, IR and so on) is changed to special values
on Lua Stack.

Also, conversion of TValue to FFI C type with store is no longer
compiled for lightuserdata.

[1]: https://www.kernel.org/doc/html/latest/arm64/memory.html

Sergey Kaplun:
* added the description and the test for the problem

Resolves tarantool/tarantool#2712
Needed for tarantool/tarantool#6154
Part of tarantool/tarantool#5629

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

test/tarantool-tests/CMakeLists.txt

@@ -57,6 +57,7 @@ macro(BuildTestCLib lib sources)
 endmacro()
 
 add_subdirectory(gh-4427-ffi-sandwich)
+add_subdirectory(lj-49-bad-lightuserdata)
 add_subdirectory(lj-flush-on-trace)
 add_subdirectory(misclib-getmetrics-capi)
 

test/tarantool-tests/lj-49-bad-lightuserdata.test.lua

@@ -0,0 +1,11 @@
+local tap = require('tap')
+
+local test = tap.test('lj-49-bad-lightuserdata')
+test:plan(2)
+
+local testlightuserdata = require('testlightuserdata')
+
+test:ok(testlightuserdata.crafted_ptr())
+test:ok(testlightuserdata.mmaped_ptr())
+
+os.exit(test:check() and 0 or 1)

test/tarantool-tests/lj-49-bad-lightuserdata/CMakeLists.txt

@@ -0,0 +1 @@
+BuildTestCLib(testlightuserdata testlightuserdata.c)

test/tarantool-tests/lj-49-bad-lightuserdata/testlightuserdata.c

@@ -0,0 +1,61 @@
+#include <lua.h>
+#include <lauxlib.h>
+
+#include <sys/mman.h>
+#include <unistd.h>
+
+#undef NDEBUG
+#include <assert.h>
+
+#define START ((void *)-1)
+
+static int crafted_ptr(lua_State *L)
+{
+	/*
+	 * We know that for arm64 at least 48 bits are available.
+	 * So emulate manually push of lightuseradata within
+	 * this range.
+	 */
+	void *longptr = (void *)(1llu << 48);
+	lua_pushlightuserdata(L, longptr);
+	assert(longptr == lua_topointer(L, -1));
+	/* Clear our stack. */
+	lua_pop(L, 0);
+	lua_pushboolean(L, 1);
+	return 1;
+}
+
+static int mmaped_ptr(lua_State *L)
+{
+	/*
+	 * If start mapping address is not NULL, then the kernel
+	 * takes it as a hint about where to place the mapping, so
+	 * we try to get the highest memory address by hint, that
+	 * equals to -1.
+	 */
+	const size_t pagesize = getpagesize();
+	void *mmaped = mmap(START, pagesize, PROT_NONE, MAP_PRIVATE | MAP_ANON,
+			    -1, 0);
+	if (mmaped != MAP_FAILED) {
+		lua_pushlightuserdata(L, mmaped);
+		assert(mmaped == lua_topointer(L, -1));
+		assert(munmap(mmaped, pagesize) == 0);
+	}
+	/* Clear our stack. */
+	lua_pop(L, 0);
+	lua_pushboolean(L, 1);
+	return 1;
+}
+
+static const struct luaL_Reg testlightuserdata[] = {
+	{"crafted_ptr", crafted_ptr},
+	{"mmaped_ptr", mmaped_ptr},
+	{NULL, NULL}
+};
+
+LUA_API int luaopen_testlightuserdata(lua_State *L)
+{
+	luaL_register(L, "testlightuserdata", testlightuserdata);
+	return 1;
+}
+