Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix issue 17562 - tan returning -nan for inputs where abs(x) >= 2^63
The fptan instruction pushes a 1.0 onto the FPU register stack after a successful operation, but when abs(input) >= 2^63 the C2 flag is set to indicate that the input was out of bounds, and it doesn't push the 1.0. Prior to this PR, the top value of the FPU stack was popped irrespective of whether C2 was set, which in the case of an out-of-bounds input caused the input to be removed from the stack and ultimately resulted in an incorrect return value of -nan. This PR changes this behavior, only popping after fptan when C2 was not set. See: http://x86.renejeschke.de/html/file_module_x86_id_109.html * Added unit tests for handling out-of-range inputs of fptan
- Loading branch information