[Bug] Ax Sweeper broken with PyTorch 1.8.0

[omry]

Ax plugin tests are failing with pytorch 1.8.
For now, I am pinning to pytorch 1.7.1.

Once solve, revert #1451

@bernardbeckerman, can you take a look? this looks very worrying, the optimization result looks flat-out wrong.

[bernardbeckerman]

the test @omry is referring to is this one. investigating now. i have some initial ideas but want to get more feedback before discussing. thanks for your patience.

[bernardbeckerman]

copying @Balandat 's comment from internal chat here:

Pretty sure this is b/c of a backward compatibility breaking (ish) change that I made to pytorch: pytorch/pytorch#49710

"ish" because it doesn't change the distributional properties of the Sobol engine, it just changes the realization of the samples.

It seems that this test is not catching a bug, but that it is testing something that is not a feature of Ax, namely, converging on an optimum after N samples (especially difficult for N=2). We can continue to discuss in our internal thread what you'd like to test, however I think the best way to proceed would be to disable the offending test and un-pin pytorch, then re-enable the test once we figure out exactly what it should look like.

[omry]

I am okay with increasing the number of iterations for this test since it seems like the proper solution based on our internal discussion.

My only requirement is that the is fast "enough" to be as a part of the standard test suite.

[omry]

https://pypi.org/project/hydra-ax-sweeper/1.1.5rc1/ no longer pins the PyTorch version.