MacOS: three distribution tests are skipped

[mruberry]

See https://app.circleci.com/pipelines/github/pytorch/pytorch/339308/workflows/261d5f9e-ed64-444f-a9ce-ff98a06d2daf/jobs/14261979 for a recent log showing the failures. #60348 has skipped these tests on MacOS to unblock the job.

These tests may just need a tolerance adjustment due to SciPy updates.

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @anjali411 @fritzo @neerajprad @alicanb @nikitaved @mruberry @VitalyFedyunin @walterddr

[malfet]

Reopening as the issue is not closed

[malfet]

I can transition test from passing to failing by changing scipy version:

$ pip install scipy==1.5.4; python test_distributions.py -v -k test_poisson_sample
Requirement already satisfied: scipy==1.5.4 in /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages (1.5.4)
Requirement already satisfied: numpy>=1.14.5 in /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages (from scipy==1.5.4) (1.18.5)
test_poisson_sample (__main__.TestDistributions) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.128s

OK
static:distributions distiller$ pip install scipy==1.7.1; python test_distributions.py -v -k test_poisson_sample
Collecting scipy==1.7.1
  Using cached scipy-1.7.1-cp37-cp37m-macosx_10_9_x86_64.whl (32.6 MB)
Requirement already satisfied: numpy<1.23.0,>=1.16.5 in /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages (from scipy==1.7.1) (1.18.5)
Installing collected packages: scipy
  Attempting uninstall: scipy
    Found existing installation: scipy 1.5.4
    Uninstalling scipy-1.5.4:
      Successfully uninstalled scipy-1.5.4
Successfully installed scipy-1.7.1
test_poisson_sample (__main__.TestDistributions) ... ERROR

======================================================================
ERROR: test_poisson_sample (__main__.TestDistributions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_distributions.py", line 1333, in test_poisson_sample
    failure_rate=1e-3)
  File "test_distributions.py", line 805, in _check_sampler_discrete
    chisq, p = scipy.stats.chisquare(counts[msk], pmf[msk] * num_samples)
  File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6853, in chisquare
    lambda_="pearson")
  File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6694, in power_divergence
    raise ValueError(msg)
ValueError: For each axis slice, the sum of the observed frequencies must agree with the sum of the expected frequencies to a relative tolerance of 1e-08, but the percent differences are:
4.535600093557479e-05

----------------------------------------------------------------------
Ran 1 test in 0.065s

FAILED (errors=1)

[malfet]

Which I guess answers following esoteric question:

pytorch/.circleci/docker/common/install_conda.sh

Line 103 in 5060b69

# TODO: Why is scipy pinned

[bdhirsh]

@malfet pointed out: these 3 tests are currently running, but they're pinned to a version of scipy that allows them to pass. We should fix the tests and not rely on the pinned version of scipy.

[malfet]

Those tests has been unskipped a while back