MAINT: Backports for 1.4.0rc2

doc/release/1.4.0-notes.rst

@@ -272,6 +272,7 @@ The documentation of many functions has been improved.
 operates on high dimensional and nonlinear data sets. It has higher statistical
 power than other `scipy.stats` tests while being the only one that operates on
 multivariate data.
+
 The generalized inverse Gaussian distribution (`scipy.stats.geninvgauss`) has 
 been added.
 
@@ -291,9 +292,6 @@ handling of ``NaN`` values
 `scipy.stats.gaussian_kde.resample` now accepts a ``seed`` argument to empower
 reproducibility
 
-`scipy.stats.multiscale_graphcorr` has been added for calculation of the
-multiscale graph correlation (MGC) test statistic
-
 `scipy.stats.kendalltau` performance has improved, especially for large inputs,
 due to improved cache usage
 
@@ -320,6 +318,11 @@ In `scipy.spatial.Rotation` methods ``from_dcm``, ``as_dcm`` were renamed to
 ``from_matrix``, ``as_matrix`` respectively. The old names will be removed in 
 SciPy 1.6.0.
 
+Method ``Rotation.match_vectors`` was deprecated in favor of 
+``Rotation.align_vectors``, which provides a more logical and 
+general API to the same functionality. The old method 
+will be removed in SciPy 1.6.0.
+
 Backwards incompatible changes
 ==============================
 
@@ -343,16 +346,10 @@ Sparse matrix reshape now raises an error if shape is not two-dimensional,
 rather than guessing what was meant. The behavior is now the same as before 
 SciPy 1.1.0.
 
-
-`scipy.spatial` changes
------------------------
-The default behavior of the ``match_vectors`` method of 
-`scipy.spatial.transform.Rotation` was changed for input vectors 
-that are not normalized and not of equal lengths.
-Previously, such vectors would be normalized within the method.  
-Now, the calculated rotation takes the vector length into account, longer 
-vectors will have a larger weight. For more details, see 
-https://github.com/scipy/scipy/issues/10968.
+``CSR`` and ``CSC`` sparse matrix classes should now return empty matrices
+of the same type when indexed out of bounds. Previously, for some versions
+of SciPy, this would raise an ``IndexError``. The change is largely motivated
+by greater consistency with ``ndarray`` and ``numpy.matrix`` semantics.
 
 `scipy.signal` changes
 ----------------------
@@ -392,6 +389,7 @@ Authors
 =======
 
 * @endolith
+* @wenhui-prudencemed +
 * Abhinav +
 * Anne Archibald
 * ashwinpathak20nov1996 +
@@ -533,7 +531,7 @@ Authors
 * Kentaro Yamamoto +
 * Dave Zbarsky +
 
-A total of 141 people contributed to this release.
+A total of 142 people contributed to this release.
 People with a "+" by their names contributed a patch for the first time.
 This list of names is automatically generated, and may not be fully complete.
 
@@ -621,6 +619,7 @@ Issues closed for 1.4.0
 * `#10588 <https://github.com/scipy/scipy/issues/10588>`__: scipy.fft and numpy.fft inconsistency when axes=None and shape...
 * `#10628 <https://github.com/scipy/scipy/issues/10628>`__: Scipy python>3.6 Windows wheels don't ship msvcp\*.dll
 * `#10733 <https://github.com/scipy/scipy/issues/10733>`__: DOC/BUG: min_only result does not match documentation
+* `#10774 <https://github.com/scipy/scipy/issues/10774>`__: min_only=true djisktra infinite loop with duplicate indices
 * `#10775 <https://github.com/scipy/scipy/issues/10775>`__: UnboundLocalError in Radau when given a NaN
 * `#10835 <https://github.com/scipy/scipy/issues/10835>`__: io.wavfile.read unnecessarily raises an error for a bad wav header
 * `#10838 <https://github.com/scipy/scipy/issues/10838>`__: Error in documentation for scipy.linalg.lu_factor
@@ -635,6 +634,9 @@ Issues closed for 1.4.0
 * `#11033 <https://github.com/scipy/scipy/issues/11033>`__: deadlock on osx for python 3.8
 * `#11041 <https://github.com/scipy/scipy/issues/11041>`__: Test failure in wheel builds for TestTf2zpk.test_simple
 * `#11089 <https://github.com/scipy/scipy/issues/11089>`__: Regression in scipy.stats where distribution will not accept loc and scale parameters
+* `#11100 <https://github.com/scipy/scipy/issues/11100>`__: BUG: multiscale_graphcorr random state seeding and parallel use
+* `#11121 <https://github.com/scipy/scipy/issues/11121>`__: Calls to `scipy.interpolate.splprep` increase RAM usage.
+* `#11125 <https://github.com/scipy/scipy/issues/11125>`__: BUG: segfault when slicing a CSR or CSC sparse matrix with slice start index > stop index
 
 Pull requests for 1.4.0
 -----------------------
@@ -995,5 +997,13 @@ Pull requests for 1.4.0
 * `#11067 <https://github.com/scipy/scipy/pull/11067>`__: DOC: updated documentation for consistency in writing style
 * `#11070 <https://github.com/scipy/scipy/pull/11070>`__: DOC: Amendment to Ubuntu Development Environment Quickstart should...
 * `#11073 <https://github.com/scipy/scipy/pull/11073>`__: DOC: fix 1.4.0 release notes
+* `#11081 <https://github.com/scipy/scipy/pull/11081>`__: API: Replace Rotation.match_vectors with align_vectors
 * `#11083 <https://github.com/scipy/scipy/pull/11083>`__: DOC: more 1.4.0 release note fixes
 * `#11092 <https://github.com/scipy/scipy/pull/11092>`__: BUG: stats: fix freezing of some distributions
+* `#11096 <https://github.com/scipy/scipy/pull/11096>`__: BUG: scipy.sparse.csgraph: fixed issue #10774
+* `#11124 <https://github.com/scipy/scipy/pull/11124>`__: fix Cython warnings related to _stats.pyx
+* `#11126 <https://github.com/scipy/scipy/pull/11126>`__: BUG: interpolate/fitpack: fix memory leak in splprep
+* `#11127 <https://github.com/scipy/scipy/pull/11127>`__: Avoid potential segfault in CSR and CSC matrix indexing
+* `#11152 <https://github.com/scipy/scipy/pull/11152>`__: BUG: Fix random state bug multiscale_graphcorr
+* `#11166 <https://github.com/scipy/scipy/pull/11166>`__: BUG: empty sparse slice shapes
+* `#11167 <https://github.com/scipy/scipy/pull/11167>`__: BUG: redundant fft in signal.resample

scipy/interpolate/src/_fitpackmodule.c

@@ -437,6 +437,8 @@ fitpack_parcur(PyObject *dummy, PyObject *args)
         }
         n = no = ap_t->dimensions[0];
         memcpy(t, ap_t->data, n*sizeof(double));
+        Py_DECREF(ap_t);
+        ap_t = NULL;
     }
     if (iopt == 1) {
         memcpy(wrk, ap_wrk->data, n*sizeof(double));
@@ -466,10 +468,18 @@ fitpack_parcur(PyObject *dummy, PyObject *args)
         goto fail;
     }
     if (iopt == 0|| n > no) {
+        Py_XDECREF(ap_wrk);
+        ap_wrk = NULL;
+        Py_XDECREF(ap_iwrk);
+        ap_iwrk = NULL;
+
         dims[0] = n;
         ap_wrk = (PyArrayObject *)PyArray_SimpleNew(1, dims, NPY_DOUBLE);
+        if (ap_wrk == NULL) {
+            goto fail;
+        }
         ap_iwrk = (PyArrayObject *)PyArray_SimpleNew(1, dims, NPY_INT);
-        if (ap_wrk == NULL || ap_iwrk == NULL) {
+        if (ap_iwrk == NULL) {
             goto fail;
         }
     }

scipy/signal/signaltools.py

@@ -2830,7 +2830,6 @@ def resample(x, num, t=None, axis=0, window=None):
     >>> plt.show()
     """
     x = np.asarray(x)
-    X = sp_fft.fft(x, axis=axis)
     Nx = x.shape[axis]
 
     # Check if we can use faster real FFT

scipy/sparse/compressed.py

@@ -782,6 +782,7 @@ def _get_submatrix(self, major=None, minor=None, copy=False):
         M, N = self._swap(self.shape)
         i0, i1 = _process_slice(major, M)
         j0, j1 = _process_slice(minor, N)
+
         if i0 == 0 and j0 == 0 and i1 == M and j1 == N:
             return self.copy() if copy else self
 
@@ -1275,6 +1276,7 @@ def _process_slice(sl, num):
         i0, i1, stride = sl.indices(num)
         if stride != 1:
             raise ValueError('slicing with step != 1 not supported')
+        i0 = min(i0, i1)  # give an empty slice when i0 > i1
     elif isintlike(sl):
         if sl < 0:
             sl += num

scipy/sparse/csgraph/_shortest_path.pyx

@@ -630,10 +630,12 @@ cdef _dijkstra_setup_heap_multi(FibonacciHeap *heap,
     heap.min_node = NULL
     for i in range(Nind):
         j_source = source_indices[i]
+        current_node = &nodes[j_source]
+        if current_node.state == SCANNED:
+            continue
         dist_matrix[j_source] = 0
         if return_pred:
             sources[j_source] = j_source
-        current_node = &nodes[j_source]
         current_node.state = SCANNED
         current_node.source = j_source
         insert_node(heap, &nodes[j_source])

scipy/sparse/csgraph/tests/test_shortest_path.py

@@ -149,7 +149,7 @@ def check(method, directed_in):
 @pytest.mark.parametrize('directed, SP_ans',
                          ((True, directed_SP),
                           (False, undirected_SP)))
-@pytest.mark.parametrize('indices', ([0, 2, 4], [0, 4], [3, 4]))
+@pytest.mark.parametrize('indices', ([0, 2, 4], [0, 4], [3, 4], [0, 0]))
 def test_dijkstra_indices_min_only(directed, SP_ans, indices):
     SP_ans = np.array(SP_ans)
     indices = np.array(indices, dtype=np.int64)

scipy/sparse/tests/test_base.py

@@ -2384,6 +2384,7 @@ def test_slicing_3(self):
 
         s_ = np.s_
         slices = [s_[:2], s_[1:2], s_[3:], s_[3::2],
+                  s_[15:20], s_[3:2],
                   s_[8:3:-1], s_[4::-2], s_[:5:-1],
                   0, 1, s_[:], s_[1:5], -1, -2, -5,
                   array(-1), np.int8(-3)]

scipy/sparse/tests/test_csc.py

@@ -4,6 +4,8 @@
 from numpy.testing import assert_array_almost_equal, assert_
 from scipy.sparse import csr_matrix, csc_matrix
 
+import pytest
+
 
 def test_csc_getrow():
     N = 10
@@ -34,3 +36,38 @@ def test_csc_getcol():
         assert_array_almost_equal(arr_col, csc_col.toarray())
         assert_(type(csc_col) is csc_matrix)
 
+@pytest.mark.parametrize("matrix_input, axis, expected_shape",
+    [(csc_matrix([[1, 0],
+                [0, 0],
+                [0, 2]]),
+      0, (0, 2)),
+     (csc_matrix([[1, 0],
+                [0, 0],
+                [0, 2]]),
+      1, (3, 0)),
+     (csc_matrix([[1, 0],
+                [0, 0],
+                [0, 2]]),
+      'both', (0, 0)),
+     (csc_matrix([[0, 1, 0, 0, 0, 0],
+                [0, 0, 0, 0, 0, 0],
+                [0, 0, 2, 3, 0, 1]]),
+      0, (0, 6))])
+def test_csc_empty_slices(matrix_input, axis, expected_shape):
+    # see gh-11127 for related discussion
+    slice_1 = matrix_input.A.shape[0] - 1
+    slice_2 = slice_1
+    slice_3 = slice_2 - 1
+
+    if axis == 0:
+        actual_shape_1 = matrix_input[slice_1:slice_2, :].A.shape
+        actual_shape_2 = matrix_input[slice_1:slice_3, :].A.shape
+    elif axis == 1:
+        actual_shape_1 = matrix_input[:, slice_1:slice_2].A.shape
+        actual_shape_2 = matrix_input[:, slice_1:slice_3].A.shape
+    elif axis == 'both':
+        actual_shape_1 = matrix_input[slice_1:slice_2, slice_1:slice_2].A.shape
+        actual_shape_2 = matrix_input[slice_1:slice_3, slice_1:slice_3].A.shape
+
+    assert actual_shape_1 == expected_shape
+    assert actual_shape_1 == actual_shape_2

scipy/sparse/tests/test_csr.py

@@ -4,6 +4,8 @@
 from numpy.testing import assert_array_almost_equal, assert_
 from scipy.sparse import csr_matrix
 
+import pytest
+
 
 def _check_csr_rowslice(i, sl, X, Xcsr):
     np_slice = X[i, sl]
@@ -58,3 +60,38 @@ def test_csr_getcol():
         assert_array_almost_equal(arr_col, csr_col.toarray())
         assert_(type(csr_col) is csr_matrix)
 
+@pytest.mark.parametrize("matrix_input, axis, expected_shape",
+    [(csr_matrix([[1, 0, 0, 0],
+                [0, 0, 0, 0],
+                [0, 2, 3, 0]]),
+      0, (0, 4)),
+     (csr_matrix([[1, 0, 0, 0],
+                [0, 0, 0, 0],
+                [0, 2, 3, 0]]),
+      1, (3, 0)),
+     (csr_matrix([[1, 0, 0, 0],
+                [0, 0, 0, 0],
+                [0, 2, 3, 0]]),
+      'both', (0, 0)),
+     (csr_matrix([[0, 1, 0, 0, 0],
+                [0, 0, 0, 0, 0],
+                [0, 0, 2, 3, 0]]),
+      0, (0, 5))])
+def test_csr_empty_slices(matrix_input, axis, expected_shape):
+    # see gh-11127 for related discussion
+    slice_1 = matrix_input.A.shape[0] - 1
+    slice_2 = slice_1
+    slice_3 = slice_2 - 1
+
+    if axis == 0:
+        actual_shape_1 = matrix_input[slice_1:slice_2, :].A.shape
+        actual_shape_2 = matrix_input[slice_1:slice_3, :].A.shape
+    elif axis == 1:
+        actual_shape_1 = matrix_input[:, slice_1:slice_2].A.shape
+        actual_shape_2 = matrix_input[:, slice_1:slice_3].A.shape
+    elif axis == 'both':
+        actual_shape_1 = matrix_input[slice_1:slice_2, slice_1:slice_2].A.shape
+        actual_shape_2 = matrix_input[slice_1:slice_3, slice_1:slice_3].A.shape
+
+    assert actual_shape_1 == expected_shape
+    assert actual_shape_1 == actual_shape_2