Inlining struggles with specific case.

[IsaacMarovitz]

I'm looking to inline a function that looks like this:

extern "C" void SOCEndianSwap32(unsigned int* ptr)
{
	if (!SOCEndianCache(ptr))
	{
		*ptr = SWAP32(*ptr);
	}
}

In practice, this gets inlined differently depending on which member of a struct this function is called on. For example on the first entry:

mov     esi, edi
call    SOCEndianCache
test    al, al
jnz     short loc_4533A7
mov     eax, [edi]
mov     ecx, eax
mov     edx, eax
shr     edx, 10h
and     ecx, offset unk_FF0000
or      ecx, edx
mov     edx, eax
shl     edx, 10h
and     eax, 0FF00h
or      edx, eax
shr     ecx, 8
shl     edx, 8
or      ecx, edx
mov     [edi], ecx

Versus a struct value at a given offset:

lea     esi, [edi+4]
call    SOCEndianCache
test    al, al
jnz     short loc_4533DA
mov     eax, [esi]
mov     ecx, eax
mov     edx, eax
shr     edx, 10h
and     ecx, offset unk_FF0000
or      ecx, edx
mov     edx, eax
shl     edx, 10h
and     eax, 0FF00h
or      edx, eax
shr     ecx, 8
shl     edx, 8
or      ecx, edx
mov     [esi], ecx

The differing lea/mov and final mov instructions cause hrtng to be unable to match these two blocks as the same functions. The plugin seems unable or unwilling to accept the section without including these starting and ending instructions.

[srgblv]

As for the final mov, it doesn't affect, de-inliner do not distinguish what exact register or even stack variable is used by instruction.

lea and mov: yea, different in assembler level. But de-inliner works with microcode, and there is no lea instruction at all.
At microcode level here should see something like: SOCEndianCache(var) and SOCEndianCache(var + 4). Where var, var + 4 or more complex expression as argument is not a part of inlined function.

This is the known problem of the first and last blocks. Part of inlined code is merged with surrounding code. That prepares argument for the call instruction in this case.
The general solution here only to define inline on basic block boundaries without code can interfere. Left part of inline outside.

Maybe to cut from mov eax, [edi] just after jnz in the beginning of inline may give better results. Probably. Unfortunately the block too small

[IsaacMarovitz]

Maybe to cut from mov eax, [edi] just after jnz in the beginning of inline may give better results. Probably. Unfortunately the block too small

Yeah it won't accept mov -> mov.

[srgblv]

Problem of the first and last blocks mentioned above should be gone after 128f397