asm_conv: Assertion `((IRType)((ir->t).irt & IRT_TYPE)) != st' failed.

[spacewander]

Hi, I have seen your email in the LuaJIT mail list. It is a good news that you are pushing LuaJIT ahead.

Several days ago, I submitted a bug report to the LuaJIT but no response is given back.
LuaJIT#524

The bug can be reproduced with the master branch of moonjit, so I think you might feel interested about it.

I have reduced the reproduce script to a few lines:

local bit = require "bit"
local ffi = require "ffi"

ffi.cdef[[
    typedef unsigned int u32;
]]
local prime = 16777619

local function prepare_hash(s)
    -- change this line to `sq = prime` eliminates the crash
    local sq = ffi.cast("u32", prime)
    local i = #s
    while i > 0 do
        -- crash when LuaJIT try to compile this while block
        sq = ffi.cast("u32", sq * sq)
        i = bit.rshift(i, 1)
    end
end

for case = 1, 1000 do
    prepare_hash("12345678")
end

When the lua_assert(irt_type(ir->t) != st) failed, irt_type(ir->t) is IRT_INT.
So maybe something wrong happens when LuaJIT generates an invalid IR CONV INT to INT, triggers assertion failure during converting IR to mcode.

[fsfod]

It looks like simplify_conv_narrow is always emitting a signed CONV even if the type its folding is U32

moonjit/src/lj_opt_fold.c

Lines 1258 to 1272 in 3f39980

	LJFOLDF(simplify_conv_narrow)
	{
	IROp op = (IROp)fleft->o;
	IRType t = irt_type(fins->t);
	IRRef op1 = fleft->op1, op2 = fleft->op2, mode = fins->op2;
	PHIBARRIER(fleft);
	op1 = emitir(IRTI(IR_CONV), op1, mode);
	op2 = emitir(IRTI(IR_CONV), op2, mode);
	fins->ot = IRT(op, t);
	fins->op1 = op1;
	fins->op2 = op2;
	return RETRYFOLD;
	}
	/* Special CSE rule for CONV. */

[siddhesh]

It looks like simplify_conv_narrow is always emitting a signed CONV even if the type its folding is U32

Thanks for the tip, that helped me find the problem immediately. The INT assumption is fine, just that the conversion mode is not updated to reflect that reality. I've posted (and will push soon) #39 that should fix this problem.

[siddhesh]

Fixed in master and v2.1 branch, so 2.1.2 should have this.

[fsfod]

Shouldn't the signedness of the IR op be preserved?

[siddhesh]

It doesn't matter for integer ADD, SUB or MUL since they don't care about signedness.

There is one tiny issue with this approach: coercing everything to INT has the side effect that simplify_conv_int_i64 will fail to see conversions of U32 -> U64 -> U32 and optimise them out in such situations since the last U64->U32 gets coerced to U64->INT in simplify_narrow_conv. So it's a missed optimization in the interest of very slightly faster compilation.

That said, in this case the conversion sequence is INT->I64->U32 and here the INT coercion helps eliminate the conversion. So this specific test case is faster. I'll open a separate issue for this.