Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Bug fix float overflow in estimate_sigma #504
The problem was that if you are really unlucky, since scipy stores the intermediate computation as the same datatype as the output array, elef managed to overflow 10**38. It's the same bug that you had with the stanford T1 overflowing int16.
Since the convolution is the sum of a 3x3x3 kernel region, it was sometime overflowing and giving back negatives values, hence the problem. Now it's up at 10**308, so I would be surprised it breaks this time.
The other arrays are fine, the problem is the way scipy stores the intermediate array when you specify one. Unless your dataset uses the full dynamic range of float32 (which is 10**38), you shouldn't have any problem.