Blend factor not working (+fix proposal)

[jeremieroy]

The new feature BGFX_STATE_BLEND_FACTOR doesn't work as it is due, to a missmatch in mask comparison (and color format issue).

In line 2402 of renderer_d3d9.cpp or 2656 of renderer_gl.cpp (need to check other renderers as well)
This is incorrect:

if (0 != (blend&(BGFX_STATE_BLEND_FUNC(BGFX_STATE_BLEND_FACTOR, BGFX_STATE_BLEND_FACTOR) ) )

the "blend" variable is a 32bit value shifted by BGFX_STATE_BLEND_SHIFT while BGFX_STATE_BLEND_FUNC is an unshifted 64 bit value, so they never match.
Furthermore, BGFX_STATE_BLEND_xxx are not mask values (power of two), they are incremental values (0,1,2,3...9, a,b...) so you cannot use binary val&masl !=0 to check for them since they will also match unwanted blend states !
Finally, BGFX_STATE_BLEND_INV_FACTOR must also be checked since it also use the blend factor.
A correct check would unfortunately be: (but there are other possible implementation)

//if (0 != (blend&(BGFX_STATE_BLEND_FUNC(BGFX_STATE_BLEND_FACTOR, BGFX_STATE_BLEND_FACTOR) ) )
if( ((src == (BGFX_STATE_BLEND_FACTOR>>BGFX_STATE_BLEND_SHIFT)) || 
     (dst == (BGFX_STATE_BLEND_FACTOR>>BGFX_STATE_BLEND_SHIFT)) ||
     (src == (BGFX_STATE_BLEND_INV_FACTOR>>BGFX_STATE_BLEND_SHIFT)) || 
     (dst == (BGFX_STATE_BLEND_INV_FACTOR>>BGFX_STATE_BLEND_SHIFT)))
&&  blendFactor != state.m_rgba)
{   

Best way to improve this would be to make every (BGFX_STATE_BLEND_xxx a power of 2. Didn't check if range allows it. The change below should be applied to every renderer, or something equivalent if the expression is optimized.

Finally the color given is in 0xRRGGBBAA, which is fine with opengl, however in direct3D (9,10,11) it must be swapped using D3DCOLOR_RGBA.
Using this :

&&  blendFactor != state.m_rgba)
{
    blendFactor = state.m_rgba;
    DWORD color = D3DCOLOR_RGBA(blendFactor>>24, (blendFactor>>16)&0xff, (blendFactor>>8)&0xff, blendFactor&0xff);
    DX_CHECK(device->SetRenderState(D3DRS_BLENDFACTOR, color) );
}

The code provided was tested locally and provide the requested result. If necessary I could make a pull request, but I think you may want to either improve the test expression, or change blend state to power of two...

[bkaradzic]

Fixed: a26e069

[jeremieroy]

Confirmed.