FOGD can Return NaN #1
Description
I was experimenting with your implementation of FOGD in this framework, and it turns out it can provide some misleading results. On this line of your approximation for sin(ux) you take a square root of 1 - cos(ux)^2. Normally this computation would be safe since the range of cos^2 is (0, 1); however, because of floating point precision, the value sometimes falls slightly outside this range, the algorithm tries to take the square root of a negative number, and returns NaN.
Looking at the implementation of your Loss Function, it appears whenever this happens, it will cause the instance to be classified as negative. This leads to improved performance on datasets with a majority of negative labels and decreased performance on majority positive label datasets. It can be seen most easily by increasing the value of gamma in your a9a example from 0.001 -> 0.01.
My approach to fixing this was clamping the value of p1 between 0 and 1 after it was calculated here.