test_kbit_quantile_estimation fails intermittently (1 in 10 runs or so - assertion error) #90
Description
Hi running this test fails about 1 in 10 times, mostly fails are .036-.038 rarely larger.
pytest ./tests/test_functional.py::test_kbit_quantile_estimation
______________________________________________________________ test_kbit_quantile_estimation _______________________________________________________________
def test_kbit_quantile_estimation():
for i in range(100):
data = torch.randn(1024, 1024, device='cuda')
for bits in range(2, 9):
p = np.linspace(1.3e-4, 1-1.3e-4, 2**bits)
val1 = torch.Tensor(norm.ppf(p)).cuda()
val2 = F.estimate_quantiles(data, offset=0, num_quantiles=2**bits)
err = torch.abs(val1-val2).mean()
> assert err < 0.035
E AssertionError: assert tensor(0.04207, device='cuda:0') < 0.035
tests/test_functional.py:2168: AssertionError
some additional fail runs results,
FAILED tests/test_functional.py::test_kbit_quantile_estimation - AssertionError: assert tensor(0.03582, device='cuda:0') < 0.035
FAILED tests/test_functional.py::test_kbit_quantile_estimation - AssertionError: assert tensor(0.03665, device='cuda:0') < 0.035
FAILED tests/test_functional.py::test_kbit_quantile_estimation - AssertionError: assert tensor(0.03681, device='cuda:0') < 0.035
Environment
WSL2 Ubuntu 20.04, conda
torch version .................... 1.14.0.dev20221107
torch cuda version ............... 11.7
nvcc version ..................... 11.8
short hash code for bitsandbytes 08fa2e7