Use custom isnan implementation to avoid HLSL optimizer issues

[Pokechu22]

This adjusts the NaN replacement logic introduced in #9928 to work around the HLSL compiler optimizing away calls to isnan, which caused that functionality to not work with ubershaders on D3D11 and D3D12 (it did work with specialized shaders, despite a warning being logged for both; that warning is also now gone). Note that the D3DCOMPILE_IEEE_STRICTNESS flag did not solve this issue, despite the warning suggesting that it might.

Suggested by @kayru and @jamiehayes.

[Pokechu22]

jamiehayes mentioned here:

Just to be pedantic, this check will also return true for +/-INF, which may or may not be what you want. If you want separate checks for NaN and INF, may I suggest the following:

bool dolphin_isnan(float f) { return (asint(f) & 0x7FFFFFFF) > 0x7F800000; }
bool dolphin_isfinite(float f) { return (asint(f) & 0x7F800000) != 0x7F800000; }

I'll replace dolphin_isnan with that.