From 933dc6cb7b3de1d8ccaf73d124d6eb95b947ed19 Mon Sep 17 00:00:00 2001 From: HyukjinKwon Date: Thu, 15 Oct 2020 09:08:14 -0700 Subject: [PATCH] [SPARK-32247][INFRA] Install and test scipy with PyPy in GitHub Actions ### What changes were proposed in this pull request? This PR proposes to install `scipy` as well in PyPy. It will test several ML specific test cases in PyPy as well. For example, https://github.com/apache/spark/blob/31a16fbb405a19dc3eb732347e0e1f873b16971d/python/pyspark/mllib/tests/test_linalg.py#L487 It was not installed when GitHub Actions build was added because it failed to install for an unknown reason. Seems like it's fixed in the latest scipy. ### Why are the changes needed? To improve test coverage. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions build in this PR will test it out. Closes #30054 from HyukjinKwon/SPARK-32247. Authored-by: HyukjinKwon Signed-off-by: Dongjoon Hyun --- .github/workflows/build_and_test.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index d1203c665010a..6afa41b9e949c 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -147,12 +147,11 @@ jobs: - name: Install Python packages (Python 2.7 and PyPy3) if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. - # TODO(SPARK-32247): scipy installation with PyPy fails for an unknown reason. run: | python2.7 -m pip install numpy 'pyarrow<3.0.0' pandas scipy xmlrunner python2.7 -m pip list # PyPy does not have xmlrunner - pypy3 -m pip install numpy pandas + pypy3 -m pip install numpy pandas scipy pypy3 -m pip list - name: Install Python packages (Python 3.8) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-'))