[svacer] lua_pcall result is ignored in src/lua/serializer.c

[sergepetrenko]

Svacer link

Relevant code:

tarantool/src/lua/serializer.c

Lines 269 to 282 in 1f128c5

	if (lua_pcall(L, 2, 1, 0) == 0 && !lua_isnil(L, -1)) {
	if (!lua_isfunction(L, -1)) {
	diag_set(LuajitError,
	"invalid " LUAL_SERIALIZE " value");
	return -1;
	}
	/* copy object itself */
	lua_pushvalue(L, idx);
	lua_pcall(L, 1, 1, 0);
	/* replace obj with the unpacked value */
	lua_replace(L, idx);
	if (luaL_tofield(L, cfg, idx, field) < 0)
	return -1;
	} /* else ignore lua_gettable exceptions */

The result of lua_pcall on line 277 is ignored. But lua_pcall() pushes the error message on the stack in case there was an error. So if we leave the lua_pcall result unchecked we then serialize the error. Basically, if __serialize metamethod returns an error ,this error will be silently serialized.

This might've been changed accidentally quite a while ago: 5519276. In this commit a lua_call wrapped with try ... catch was replaced with a lua_pcall. So previously the error wasn't serialized, even though it was ignored as well.

It seems we shouldn't ignore the __serialize errors and report them to the user.