Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NetCDF reading results in a segfault #3630

Closed
swails opened this issue May 8, 2014 · 9 comments · Fixed by #3960
Closed

NetCDF reading results in a segfault #3630

swails opened this issue May 8, 2014 · 9 comments · Fixed by #3960
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io
Milestone

Comments

@swails
Copy link

swails commented May 8, 2014

I'm seeing a segfault when trying to read a variable out of a NetCDF file. The code necessary to reproduce this is:

$ python -c 'import scipy.io; print scipy.io.netcdf.netcdf_file("MDTraj/testing/reference/mdcrd.nc").variables["coordinates"][:].sum()'
Segmentation fault

The MDTraj/testing/reference/mdcrd.nc file is available in the MDTraj github repository (see issue rmcgibbo/mdtraj#440). You can download just the NetCDF file here: https://github.com/rmcgibbo/mdtraj/blob/master/MDTraj/testing/reference/mdcrd.nc (I don't see a way to upload a file here).

This segfaults with the current HEAD here and scipy v0.14, but works fine with scipy 0.13.x. I've used numpy 1.8.0 in all tests.

@swails
Copy link
Author

swails commented May 8, 2014

More strangeness here:

The following works for the current HEAD

$ python -c 'import scipy.io; traj=scipy.io.netcdf.netcdf_file("MDTraj/testing/reference/mdcrd.nc"); print traj.variables["coordinates"][:].sum()'
-1543.51

The key difference here being that I assigned the netcdf_file to traj and then tried accessing the variables attribute from traj. I don't know why one would work and the other would not.

@pv
Copy link
Member

pv commented May 8, 2014

Apparently crashes due to use of memmap after closing file. Workaround: use with netcdf_file(...) as f: ... or mmap=False.

@swails
Copy link
Author

swails commented May 8, 2014

I can verify that mmap=False solves our problems, thanks! Will this have any adverse performance implications for reading NetCDF files (either memory consumption or running time)? The NetCDF files are potentially very big (on the order of GB to 10s of GB).

@gauteh
Copy link
Contributor

gauteh commented May 14, 2014

Can confirm that this started happening with a recent scipy upgrade.

@andreas-h
Copy link
Contributor

I don't see how this is a defect. Maybe we should just add a warning to the docstring?

@rgommers
Copy link
Member

@andreas-h why not, I think we simply broke something in 0.14.0 that used to work.

@andreas-h
Copy link
Contributor

@rgommers Well, it seems like we actually fixed something in 0.14.0. Before then, the mmap object was not closed when the netCDF file was closed (see https://github.com/scipy/scipy/blob/master/scipy/io/netcdf.py#L247). I believe it is actually cleaner to do it like it is now.

@rgommers
Copy link
Member

Yeah, we fixed another bug. But now we need to fix this one too, without reintroducing the other one.

pv added a commit to pv/scipy-work that referenced this issue Aug 31, 2014
pv added a commit to pv/scipy-work that referenced this issue Sep 3, 2014
pv added a commit to pv/scipy-work that referenced this issue Sep 3, 2014
pv added a commit to pv/scipy-work that referenced this issue Sep 3, 2014
pv added a commit to pv/scipy-work that referenced this issue Sep 3, 2014
This reverts commit 2dfffc7.

Closing mmap fd's can cause segfaults, as discussed in scipygh-3630
@rgommers
Copy link
Member

rgommers commented Sep 5, 2014

Offending commit reverted in gh-3961. Will be fixed differently for 0.15.0, see gh-3960

martinclaus added a commit to martinclaus/cdo-bindings that referenced this issue Oct 10, 2014
… with cdfMod=CDF_MOD_SCIPY and scipy v0.14.

However, when using returnCdf together with an direct access to variable data, like `cdo.stdatm("0",options="-f nc",returnCdf=True).variables["T"][:]` there will
still be an segmentation fault due to a bug in scipy.io.netcdf.netcdf_file (see scipy/scipy#3630). In this case one will have to store the filehandle and then access the data:
`cdf = cdo.stdatm("0",options="-f nc",returnCdf=True); data = cdf.variables["T"][:]`
jsonn pushed a commit to jsonn/pkgsrc that referenced this issue Apr 17, 2015
Upstream changes:
SciPy 0.15.1 is a bug-fix release with no new features compared to 0.15.0.

Issues fixed
- ------------

* `#4413 <https://github.com/scipy/scipy/pull/4413>`__: BUG: Tests too strict, f2py doesn't have to overwrite this array
* `#4417 <https://github.com/scipy/scipy/pull/4417>`__: BLD: avoid using NPY_API_VERSION to check not using deprecated...
* `#4418 <https://github.com/scipy/scipy/pull/4418>`__: Restore and deprecate scipy.linalg.calc_work

SciPy 0.15.0 Release Notes
==========================

.. contents::

SciPy 0.15.0 is the culmination of 6 months of hard work. It contains
several new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.16.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater.


New features
============

Linear Programming Interface
- ----------------------------

The new function `scipy.optimize.linprog` provides a generic
linear programming similar to the way `scipy.optimize.minimize`
provides a generic interface to nonlinear programming optimizers.
Currently the only method supported is *simplex* which provides
a two-phase, dense-matrix-based simplex algorithm. Callbacks
functions are supported, allowing the user to monitor the progress
of the algorithm.

Differential evolution, a global optimizer
- ------------------------------------------

A new `scipy.optimize.differential_evolution` function has been added to the
``optimize`` module.  Differential Evolution is an algorithm used for finding
the global minimum of multivariate functions. It is stochastic in nature (does
not use gradient methods), and can search large areas of candidate space, but
often requires larger numbers of function evaluations than conventional
gradient based techniques.

``scipy.signal`` improvements
- -----------------------------

The function `scipy.signal.max_len_seq` was added, which computes a Maximum
Length Sequence (MLS) signal.

``scipy.integrate`` improvements
- --------------------------------

It is now possible to use `scipy.integrate` routines to integrate
multivariate ctypes functions, thus avoiding callbacks to Python and
providing better performance.

``scipy.linalg`` improvements
- -----------------------------

The function `scipy.linalg.orthogonal_procrustes` for solving the procrustes
linear algebra problem was added.

BLAS level 2 functions ``her``, ``syr``, ``her2`` and ``syr2`` are now wrapped
in ``scipy.linalg``.

``scipy.sparse`` improvements
- -----------------------------

`scipy.sparse.linalg.svds` can now take a ``LinearOperator`` as its main input.

``scipy.special`` improvements
- ------------------------------

Values of ellipsoidal harmonic (i.e. Lame) functions and associated
normalization constants can be now computed using ``ellip_harm``,
``ellip_harm_2``, and ``ellip_normal``.

New convenience functions ``entr``, ``rel_entr`` ``kl_div``,
``huber``, and ``pseudo_huber`` were added.

``scipy.sparse.csgraph`` improvements
- -------------------------------------

Routines ``reverse_cuthill_mckee`` and ``maximum_bipartite_matching``
for computing reorderings of sparse graphs were added.

``scipy.stats`` improvements
- ----------------------------

Added a Dirichlet multivariate distribution, `scipy.stats.dirichlet`.

The new function `scipy.stats.median_test` computes Mood's median test.

The new function `scipy.stats.combine_pvalues` implements Fisher's
and Stouffer's methods for combining p-values.

`scipy.stats.describe` returns a namedtuple rather than a tuple, allowing
users to access results by index or by name.


Deprecated features
===================

The `scipy.weave` module is deprecated.  It was the only module never ported
to Python 3.x, and is not recommended to be used for new code - use Cython
instead.  In order to support existing code, ``scipy.weave`` has been packaged
separately: https://github.com/scipy/weave.  It is a pure Python package, and
can easily be installed with ``pip install weave``.

`scipy.special.bessel_diff_formula` is deprecated.  It is a private function,
and therefore will be removed from the public API in a following release.

``scipy.stats.nanmean``, ``nanmedian`` and ``nanstd`` functions are deprecated
in favor of their numpy equivalents.


Backwards incompatible changes
==============================

scipy.ndimage
- -------------

The functions `scipy.ndimage.minimum_positions`,
`scipy.ndimage.maximum_positions`` and `scipy.ndimage.extrema` return
positions as ints instead of floats.

scipy.integrate
- ---------------

The format of banded Jacobians in `scipy.integrate.ode` solvers is
changed. Note that the previous documentation of this feature was
erroneous.

SciPy 0.14.1 Release Notes
==========================

SciPy 0.14.1 is a bug-fix release with no new features compared to 0.14.0.


Issues closed
- -------------

- - `#3630 <https://github.com/scipy/scipy/issues/3630>`__: NetCDF reading results in a segfault
- - `#3631 <https://github.com/scipy/scipy/issues/3631>`__: SuperLU object not working as expected for complex matrices
- - `#3733 <https://github.com/scipy/scipy/issues/3733>`__: segfault from map_coordinates
- - `#3780 <https://github.com/scipy/scipy/issues/3780>`__: Segfault when using CSR/CSC matrix and uint32/uint64
- - `#3781 <https://github.com/scipy/scipy/pull/3781>`__: BUG: sparse: fix omitted types in sparsetools typemaps
- - `#3802 <https://github.com/scipy/scipy/issues/3802>`__: 0.14.0 API breakage: _gen generators are missing from scipy.stats.distributions API
- - `#3805 <https://github.com/scipy/scipy/issues/3805>`__: ndimage test failures with numpy 1.10
- - `#3812 <https://github.com/scipy/scipy/issues/3812>`__: == sometimes wrong on csr_matrix
- - `#3853 <https://github.com/scipy/scipy/issues/3853>`__: Many scipy.sparse test errors/failures with numpy 1.9.0b2
- - `#4084 <https://github.com/scipy/scipy/pull/4084>`__: fix exception declarations for Cython 0.21.1 compatibility
- - `#4093 <https://github.com/scipy/scipy/pull/4093>`__: BUG: fitpack: avoid a memory error in splev(x, tck, der=k)
- - `#4104 <https://github.com/scipy/scipy/pull/4104>`__: BUG: Workaround SGEMV segfault in Accelerate (maintenance 0.14.x)
- - `#4143 <https://github.com/scipy/scipy/pull/4143>`__: BUG: fix ndimage functions for large data
- - `#4149 <https://github.com/scipy/scipy/issues/4149>`__: Bug in expm for integer arrays
- - `#4154 <https://github.com/scipy/scipy/issues/4154>`__: Backport gh-4041 for 0.14.1 (Ensure that the 'size' argument of PIL's 'resize' method is a tuple)
- - `#4163 <https://github.com/scipy/scipy/issues/4163>`__: Backport #4142 (ZeroDivisionError in scipy.sparse.linalg.lsqr)
- - `#4164 <https://github.com/scipy/scipy/issues/4164>`__: Backport gh-4153 (remove use of deprecated numpy API in lib/lapack/ f2py wrapper)
- - `#4180 <https://github.com/scipy/scipy/pull/4180>`__: backport pil resize support tuple fix
- - `#4168 <https://github.com/scipy/scipy/issues/4168>`__: Lots of arpack test failures on windows 32 bits with numpy 1.9.1
- - `#4203 <https://github.com/scipy/scipy/issues/4203>`__: Matrix multiplication in 0.14.x is more than 10x slower compared...
- - `#4218 <https://github.com/scipy/scipy/pull/4218>`__: attempt to make ndimage interpolation compatible with numpy relaxed...
- - `#4225 <https://github.com/scipy/scipy/pull/4225>`__: BUG: off-by-one error in PPoly shape checks
- - `#4248 <https://github.com/scipy/scipy/pull/4248>`__: BUG: optimize: fix issue with incorrect use of closure for slsqp.

SciPy 0.14.0 Release Notes
==========================

.. contents::

SciPy 0.14.0 is the culmination of 8 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.14.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater.


New features
============

``scipy.interpolate`` improvements
----------------------------------

A new wrapper function `scipy.interpolate.interpn` for interpolation on regular
grids has been added. `interpn` supports linear and nearest-neighbor
interpolation in arbitrary dimensions and spline interpolation in two
dimensions.

Faster implementations of piecewise polynomials in power and Bernstein
polynomial bases have been added as `scipy.interpolate.PPoly` and
`scipy.interpolate.BPoly`. New users should use these in favor of
`scipy.interpolate.PiecewisePolynomial`.

`scipy.interpolate.interp1d` now accepts non-monotonic inputs and sorts them.
If performance is critical, sorting can be turned off by using the new
``assume_sorted`` keyword.

Functionality for evaluation of bivariate spline derivatives in
``scipy.interpolate`` has been added.

The new class `scipy.interpolate.Akima1DInterpolator` implements the piecewise
cubic polynomial interpolation scheme devised by H. Akima.

Functionality for fast interpolation on regular, unevenly spaced grids
in arbitrary dimensions has been added as
`scipy.interpolate.RegularGridInterpolator` .


``scipy.linalg`` improvements
-----------------------------

The new function `scipy.linalg.dft` computes the matrix of the
discrete Fourier transform.

A condition number estimation function for matrix exponential,
`scipy.linalg.expm_cond`, has been added.


``scipy.optimize`` improvements
-------------------------------

A set of benchmarks for optimize, which can be run with ``optimize.bench()``,
has been added.

`scipy.optimize.curve_fit` now has more controllable error estimation via the
``absolute_sigma`` keyword.

Support for passing custom minimization methods to ``optimize.minimize()``
and ``optimize.minimize_scalar()`` has been added, currently useful especially
for combining ``optimize.basinhopping()`` with custom local optimizer routines.


``scipy.stats`` improvements
----------------------------

A new class `scipy.stats.multivariate_normal` with functionality for
multivariate normal random variables has been added.

A lot of work on the ``scipy.stats`` distribution framework has been done.
Moment calculations (skew and kurtosis mainly) are fixed and verified, all
examples are now runnable, and many small accuracy and performance improvements
for individual distributions were merged.

The new function `scipy.stats.anderson_ksamp` computes the k-sample
Anderson-Darling test for the null hypothesis that k samples come from
the same parent population.


``scipy.signal`` improvements
-----------------------------

``scipy.signal.iirfilter`` and related functions to design Butterworth,
Chebyshev, elliptical and Bessel IIR filters now all use pole-zero ("zpk")
format internally instead of using transformations to numerator/denominator
format.  The accuracy of the produced filters, especially high-order ones, is
improved significantly as a result.

The new function `scipy.signal.vectorstrength` computes the vector strength,
a measure of phase synchrony, of a set of events.


``scipy.special`` improvements
------------------------------

The functions `scipy.special.boxcox` and `scipy.special.boxcox1p`, which
compute the Box-Cox transformation, have been added.


``scipy.sparse`` improvements
-----------------------------

- Significant performance improvement in CSR, CSC, and DOK indexing speed.
- When using Numpy >= 1.9 (to be released in MM 2014), sparse matrices function
  correctly when given to arguments of ``np.dot``, ``np.multiply`` and other
  ufuncs.  With earlier Numpy and Scipy versions, the results of such
  operations are undefined and usually unexpected.
- Sparse matrices are no longer limited to ``2^31`` nonzero elements.  They
  automatically switch to using 64-bit index data type for matrices containing
  more elements.  User code written assuming the sparse matrices use int32 as
  the index data type will continue to work, except for such large matrices.
  Code dealing with larger matrices needs to accept either int32 or int64
  indices.


Deprecated features
===================

``anneal``
----------

The global minimization function `scipy.optimize.anneal` is deprecated.
All users should use the `scipy.optimize.basinhopping` function instead.

``scipy.stats``
---------------

``randwcdf`` and ``randwppf`` functions are deprecated. All users should use
distribution-specific ``rvs`` methods instead.

Probability calculation aliases ``zprob``, ``fprob`` and ``ksprob`` are
deprecated. Use instead the ``sf`` methods of the corresponding distributions
or the ``special`` functions directly.

``scipy.interpolate``
---------------------

``PiecewisePolynomial`` class is deprecated.


Backwards incompatible changes
==============================

scipy.special.lpmn
------------------

``lpmn`` no longer accepts complex-valued arguments. A new function
``clpmn`` with uniform complex analytic behavior has been added, and
it should be used instead.

scipy.sparse.linalg
-------------------

Eigenvectors in the case of generalized eigenvalue problem are normalized to
unit vectors in 2-norm, rather than following the LAPACK normalization
convention.

The deprecated UMFPACK wrapper in ``scipy.sparse.linalg`` has been removed due
to license and install issues.  If available, ``scikits.umfpack`` is still used
transparently in the ``spsolve`` and ``factorized`` functions.  Otherwise,
SuperLU is used instead in these functions.

scipy.stats
-----------

The deprecated functions ``glm``, ``oneway`` and ``cmedian`` have been removed
from ``scipy.stats``.

``stats.scoreatpercentile`` now returns an array instead of a list of
percentiles.

scipy.interpolate
-----------------

The API for computing derivatives of a monotone piecewise interpolation has
changed: if `p` is a ``PchipInterpolator`` object, `p.derivative(der)`
returns a callable object representing the derivative of `p`. For in-place
derivatives use the second argument of the `__call__` method:
`p(0.1, der=2)` evaluates the second derivative of `p` at `x=0.1`.

The method `p.derivatives` has been removed.

SciPy 0.13.3 Release Notes

SciPy 0.13.3 is a bug-fix release with no new features compared to 0.13.2. Both the weave and the ndimage.label bugs were severe regressions in 0.13.0, hence this release.
Issues fixed

    3148: fix a memory leak in ndimage.label.
    3216: fix weave issue with too long file names for MSVC.

Other changes

    Update Sphinx theme used for html docs so >>> in examples can be toggled.

SciPy 0.13.2 Release Notes

SciPy 0.13.2 is a bug-fix release with no new features compared to 0.13.1.
Issues fixed

    3096: require Cython 0.19, earlier versions have memory leaks in fused types
    3079: ndimage.label fix swapped 64-bitness test
    3108: optimize.fmin_slsqp constraint violation

SciPy 0.13.1 Release Notes

SciPy 0.13.1 is a bug-fix release with no new features compared to 0.13.0. The only changes are several fixes in ndimage, one of which was a serious regression in ndimage.label (Github issue 3025), which gave incorrect results in 0.13.0.
Issues fixed

    3025: ndimage.label returns incorrect results in scipy 0.13.0
    1992: ndimage.label return type changed from int32 to uint32
    1992: ndimage.find_objects doesn't work with int32 input in some cases

SciPy 0.13.0 Release Notes
==========================

.. contents::

SciPy 0.13.0 is the culmination of 7 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.13.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater.
Highlights of this release are:

  - support for fancy indexing and boolean comparisons with sparse matrices
  - interpolative decompositions and matrix functions in the linalg module
  - two new trust-region solvers for unconstrained minimization


New features
============

``scipy.integrate`` improvements
--------------------------------

N-dimensional numerical integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A new function `scipy.integrate.nquad`, which provides N-dimensional
integration functionality with a more flexible interface than ``dblquad`` and
``tplquad``, has been added.

``dopri*`` improvements
^^^^^^^^^^^^^^^^^^^^^^^

The intermediate results from the ``dopri`` family of ODE solvers can now be
accessed by a *solout* callback function.


``scipy.linalg`` improvements
-----------------------------

Interpolative decompositions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Scipy now includes a new module `scipy.linalg.interpolative`
containing routines for computing interpolative matrix decompositions
(ID). This feature is based on the ID software package by
P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously
adapted for Python in the PymatrixId package by K.L. Ho.

Polar decomposition
^^^^^^^^^^^^^^^^^^^

A new function `scipy.linalg.polar`, to compute the polar decomposition
of a matrix, was added.

BLAS level 3 functions
^^^^^^^^^^^^^^^^^^^^^^

The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and
``her2k`` are now wrapped in `scipy.linalg`.

Matrix functions
^^^^^^^^^^^^^^^^

Several matrix function algorithms have been implemented or updated following
detailed descriptions in recent papers of Nick Higham and his co-authors.
These include the matrix square root (``sqrtm``), the matrix logarithm
(``logm``), the matrix exponential (``expm``) and its Frechet derivative
(``expm_frechet``), and fractional matrix powers (``fractional_matrix_power``).


``scipy.optimize`` improvements
-------------------------------

Trust-region unconstrained minimization algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``minimize`` function gained two trust-region solvers for unconstrained
minimization: ``dogleg`` and ``trust-ncg``.


``scipy.sparse`` improvements
-----------------------------

Boolean comparisons and sparse matrices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

All sparse matrix types now support boolean data, and boolean operations.  Two
sparse matrices `A` and `B` can be compared in all the expected ways `A < B`,
`A >= B`, `A != B`, producing similar results as dense Numpy arrays.
Comparisons with dense matrices and scalars are also supported.

CSR and CSC fancy indexing
^^^^^^^^^^^^^^^^^^^^^^^^^^

Compressed sparse row and column sparse matrix types now support fancy indexing
with boolean matrices, slices, and lists. So where A is a (CSC or CSR) sparse
matrix, you can do things like::

    >>> A[A > 0.5] = 1  # since Boolean sparse matrices work
    >>> A[:2, :3] = 2
    >>> A[[1,2], 2] = 3


``scipy.sparse.linalg`` improvements
------------------------------------

The new function ``onenormest`` provides a lower bound of the 1-norm of a
linear operator and has been implemented according to Higham and Tisseur
(2000).  This function is not only useful for sparse matrices, but can also be
used to estimate the norm of products or powers of dense matrices without
explictly building the intermediate matrix.

The multiplicative action of the matrix exponential of a linear operator
(``expm_multiply``) has been implemented following the description in Al-Mohy
and Higham (2011).

Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can now be
multiplied, added to each other, and exponentiated, producing new linear
operators. This enables easier construction of composite linear operations.


``scipy.spatial`` improvements
------------------------------

The vertices of a `ConvexHull` can now be accessed via the `vertices` attribute,
which gives proper orientation in 2-D.


``scipy.signal`` improvements
-----------------------------

The cosine window function `scipy.signal.cosine` was added.


``scipy.special`` improvements
------------------------------

New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were added.
These functions can simplify and speed up code that has to calculate
``x * log(y)`` and give 0 when ``x == 0``.


``scipy.io`` improvements
-------------------------

Unformatted Fortran file reader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The new class `scipy.io.FortranFile` facilitates reading unformatted
sequential files written by Fortran code.

``scipy.io.wavfile`` enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.io.wavfile.write` now accepts a file buffer. Previously it only
accepted a filename.

`scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle floating
point WAV files.


``scipy.interpolate`` improvements
----------------------------------

B-spline derivatives and antiderivatives
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.interpolate.splder` and `scipy.interpolate.splantider` functions
for computing B-splines that represent derivatives and antiderivatives
of B-splines were added.  These functions are also available in the
class-based FITPACK interface as ``UnivariateSpline.derivative`` and
``UnivariateSpline.antiderivative``.


``scipy.stats`` improvements
----------------------------

Distributions now allow using keyword parameters in addition to
positional parameters in all methods.

The function `scipy.stats.power_divergence` has been added for the
Cressie-Read power divergence statistic and goodness of fit test.
Included in this family of statistics is the "G-test"
(http://en.wikipedia.org/wiki/G-test).

`scipy.stats.mood` now accepts multidimensional input.

An option was added to `scipy.stats.wilcoxon` for continuity correction.

`scipy.stats.chisquare` now has an `axis` argument.

`scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments.


Deprecated features
===================

``expm2`` and ``expm3``
-----------------------

The matrix exponential functions `scipy.linalg.expm2` and `scipy.linalg.expm3`
are deprecated. All users should use the numerically more robust
`scipy.linalg.expm` function instead.

``scipy.stats`` functions
-------------------------

`scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used
instead.

`scipy.stats.glm` is deprecated.  `scipy.stats.ttest_ind` is an equivalent
function; more full-featured general (and generalized) linear model
implementations can be found in statsmodels.

`scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used instead.


Backwards incompatible changes
==============================

LIL matrix assignment
---------------------
Assigning values to LIL matrices with two index arrays now works similarly as
assigning into ndarrays::

    >>> x = lil_matrix((3, 3))
    >>> x[[0,1,2],[0,1,2]]=[0,1,2]
    >>> x.todense()
    matrix([[ 0.,  0.,  0.],
            [ 0.,  1.,  0.],
            [ 0.,  0.,  2.]])

rather than giving the result::

    >>> x.todense()
    matrix([[ 0.,  1.,  2.],
            [ 0.,  1.,  2.],
            [ 0.,  1.,  2.]])

Users relying on the previous behavior will need to revisit their code.
The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = ...`.


Deprecated ``radon`` function removed
-------------------------------------

The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has been
removed.  Users can find a more full-featured ``radon`` function in
scikit-image.


Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions``
--------------------------------------------------------------------------

The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have
been removed from the distributions in ``scipy.stats``.

Changes to MATLAB file readers / writers
----------------------------------------

The major change is that 1D arrays in numpy now become row vectors (shape 1, N)
when saved to a MATLAB 5 format file.  Previously 1D arrays saved as column
vectors (N, 1).  This is to harmonize the behavior of writing MATLAB 4 and 5
formats, and adapt to the defaults of numpy and MATLAB - for example
``np.atleast_2d`` returns 1D arrays as row vectors.

Trying to save arrays of greater than 2 dimensions in MATLAB 4 format now raises
an error instead of silently reshaping the array as 2D.

``scipy.io.loadmat('afile')`` used to look for `afile` on the Python system path
(``sys.path``); now ``loadmat`` only looks in the current directory for a
relative path filename.


Other changes
=============

Security fix: ``scipy.weave`` previously used temporary directories in an
insecure manner under certain circumstances.

Cython is now required to build *unreleased* versions of scipy.
The C files generated from Cython sources are not included in the git repo
anymore.  They are however still shipped in source releases.

The code base received a fairly large PEP8 cleanup.  A ``tox pep8``
command has been added; new code should pass this test command.

Scipy cannot be compiled with gfortran 4.1 anymore (at least on RH5), likely
due to that compiler version not supporting entry constructs well.
jsonn pushed a commit to jsonn/pkgsrc that referenced this issue Apr 22, 2015
Upstream changes:
SciPy 0.15.1 is a bug-fix release with no new features compared to 0.15.0.

Issues fixed
- ------------

* `#4413 <https://github.com/scipy/scipy/pull/4413>`__: BUG: Tests too strict, f2py doesn't have to overwrite this array
* `#4417 <https://github.com/scipy/scipy/pull/4417>`__: BLD: avoid using NPY_API_VERSION to check not using deprecated...
* `#4418 <https://github.com/scipy/scipy/pull/4418>`__: Restore and deprecate scipy.linalg.calc_work

SciPy 0.15.0 Release Notes
==========================

.. contents::

SciPy 0.15.0 is the culmination of 6 months of hard work. It contains
several new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.16.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater.


New features
============

Linear Programming Interface
- ----------------------------

The new function `scipy.optimize.linprog` provides a generic
linear programming similar to the way `scipy.optimize.minimize`
provides a generic interface to nonlinear programming optimizers.
Currently the only method supported is *simplex* which provides
a two-phase, dense-matrix-based simplex algorithm. Callbacks
functions are supported, allowing the user to monitor the progress
of the algorithm.

Differential evolution, a global optimizer
- ------------------------------------------

A new `scipy.optimize.differential_evolution` function has been added to the
``optimize`` module.  Differential Evolution is an algorithm used for finding
the global minimum of multivariate functions. It is stochastic in nature (does
not use gradient methods), and can search large areas of candidate space, but
often requires larger numbers of function evaluations than conventional
gradient based techniques.

``scipy.signal`` improvements
- -----------------------------

The function `scipy.signal.max_len_seq` was added, which computes a Maximum
Length Sequence (MLS) signal.

``scipy.integrate`` improvements
- --------------------------------

It is now possible to use `scipy.integrate` routines to integrate
multivariate ctypes functions, thus avoiding callbacks to Python and
providing better performance.

``scipy.linalg`` improvements
- -----------------------------

The function `scipy.linalg.orthogonal_procrustes` for solving the procrustes
linear algebra problem was added.

BLAS level 2 functions ``her``, ``syr``, ``her2`` and ``syr2`` are now wrapped
in ``scipy.linalg``.

``scipy.sparse`` improvements
- -----------------------------

`scipy.sparse.linalg.svds` can now take a ``LinearOperator`` as its main input.

``scipy.special`` improvements
- ------------------------------

Values of ellipsoidal harmonic (i.e. Lame) functions and associated
normalization constants can be now computed using ``ellip_harm``,
``ellip_harm_2``, and ``ellip_normal``.

New convenience functions ``entr``, ``rel_entr`` ``kl_div``,
``huber``, and ``pseudo_huber`` were added.

``scipy.sparse.csgraph`` improvements
- -------------------------------------

Routines ``reverse_cuthill_mckee`` and ``maximum_bipartite_matching``
for computing reorderings of sparse graphs were added.

``scipy.stats`` improvements
- ----------------------------

Added a Dirichlet multivariate distribution, `scipy.stats.dirichlet`.

The new function `scipy.stats.median_test` computes Mood's median test.

The new function `scipy.stats.combine_pvalues` implements Fisher's
and Stouffer's methods for combining p-values.

`scipy.stats.describe` returns a namedtuple rather than a tuple, allowing
users to access results by index or by name.


Deprecated features
===================

The `scipy.weave` module is deprecated.  It was the only module never ported
to Python 3.x, and is not recommended to be used for new code - use Cython
instead.  In order to support existing code, ``scipy.weave`` has been packaged
separately: https://github.com/scipy/weave.  It is a pure Python package, and
can easily be installed with ``pip install weave``.

`scipy.special.bessel_diff_formula` is deprecated.  It is a private function,
and therefore will be removed from the public API in a following release.

``scipy.stats.nanmean``, ``nanmedian`` and ``nanstd`` functions are deprecated
in favor of their numpy equivalents.


Backwards incompatible changes
==============================

scipy.ndimage
- -------------

The functions `scipy.ndimage.minimum_positions`,
`scipy.ndimage.maximum_positions`` and `scipy.ndimage.extrema` return
positions as ints instead of floats.

scipy.integrate
- ---------------

The format of banded Jacobians in `scipy.integrate.ode` solvers is
changed. Note that the previous documentation of this feature was
erroneous.

SciPy 0.14.1 Release Notes
==========================

SciPy 0.14.1 is a bug-fix release with no new features compared to 0.14.0.


Issues closed
- -------------

- - `#3630 <https://github.com/scipy/scipy/issues/3630>`__: NetCDF reading results in a segfault
- - `#3631 <https://github.com/scipy/scipy/issues/3631>`__: SuperLU object not working as expected for complex matrices
- - `#3733 <https://github.com/scipy/scipy/issues/3733>`__: segfault from map_coordinates
- - `#3780 <https://github.com/scipy/scipy/issues/3780>`__: Segfault when using CSR/CSC matrix and uint32/uint64
- - `#3781 <https://github.com/scipy/scipy/pull/3781>`__: BUG: sparse: fix omitted types in sparsetools typemaps
- - `#3802 <https://github.com/scipy/scipy/issues/3802>`__: 0.14.0 API breakage: _gen generators are missing from scipy.stats.distributions API
- - `#3805 <https://github.com/scipy/scipy/issues/3805>`__: ndimage test failures with numpy 1.10
- - `#3812 <https://github.com/scipy/scipy/issues/3812>`__: == sometimes wrong on csr_matrix
- - `#3853 <https://github.com/scipy/scipy/issues/3853>`__: Many scipy.sparse test errors/failures with numpy 1.9.0b2
- - `#4084 <https://github.com/scipy/scipy/pull/4084>`__: fix exception declarations for Cython 0.21.1 compatibility
- - `#4093 <https://github.com/scipy/scipy/pull/4093>`__: BUG: fitpack: avoid a memory error in splev(x, tck, der=k)
- - `#4104 <https://github.com/scipy/scipy/pull/4104>`__: BUG: Workaround SGEMV segfault in Accelerate (maintenance 0.14.x)
- - `#4143 <https://github.com/scipy/scipy/pull/4143>`__: BUG: fix ndimage functions for large data
- - `#4149 <https://github.com/scipy/scipy/issues/4149>`__: Bug in expm for integer arrays
- - `#4154 <https://github.com/scipy/scipy/issues/4154>`__: Backport gh-4041 for 0.14.1 (Ensure that the 'size' argument of PIL's 'resize' method is a tuple)
- - `#4163 <https://github.com/scipy/scipy/issues/4163>`__: Backport #4142 (ZeroDivisionError in scipy.sparse.linalg.lsqr)
- - `#4164 <https://github.com/scipy/scipy/issues/4164>`__: Backport gh-4153 (remove use of deprecated numpy API in lib/lapack/ f2py wrapper)
- - `#4180 <https://github.com/scipy/scipy/pull/4180>`__: backport pil resize support tuple fix
- - `#4168 <https://github.com/scipy/scipy/issues/4168>`__: Lots of arpack test failures on windows 32 bits with numpy 1.9.1
- - `#4203 <https://github.com/scipy/scipy/issues/4203>`__: Matrix multiplication in 0.14.x is more than 10x slower compared...
- - `#4218 <https://github.com/scipy/scipy/pull/4218>`__: attempt to make ndimage interpolation compatible with numpy relaxed...
- - `#4225 <https://github.com/scipy/scipy/pull/4225>`__: BUG: off-by-one error in PPoly shape checks
- - `#4248 <https://github.com/scipy/scipy/pull/4248>`__: BUG: optimize: fix issue with incorrect use of closure for slsqp.

SciPy 0.14.0 Release Notes
==========================

.. contents::

SciPy 0.14.0 is the culmination of 8 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.14.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater.


New features
============

``scipy.interpolate`` improvements
----------------------------------

A new wrapper function `scipy.interpolate.interpn` for interpolation on regular
grids has been added. `interpn` supports linear and nearest-neighbor
interpolation in arbitrary dimensions and spline interpolation in two
dimensions.

Faster implementations of piecewise polynomials in power and Bernstein
polynomial bases have been added as `scipy.interpolate.PPoly` and
`scipy.interpolate.BPoly`. New users should use these in favor of
`scipy.interpolate.PiecewisePolynomial`.

`scipy.interpolate.interp1d` now accepts non-monotonic inputs and sorts them.
If performance is critical, sorting can be turned off by using the new
``assume_sorted`` keyword.

Functionality for evaluation of bivariate spline derivatives in
``scipy.interpolate`` has been added.

The new class `scipy.interpolate.Akima1DInterpolator` implements the piecewise
cubic polynomial interpolation scheme devised by H. Akima.

Functionality for fast interpolation on regular, unevenly spaced grids
in arbitrary dimensions has been added as
`scipy.interpolate.RegularGridInterpolator` .


``scipy.linalg`` improvements
-----------------------------

The new function `scipy.linalg.dft` computes the matrix of the
discrete Fourier transform.

A condition number estimation function for matrix exponential,
`scipy.linalg.expm_cond`, has been added.


``scipy.optimize`` improvements
-------------------------------

A set of benchmarks for optimize, which can be run with ``optimize.bench()``,
has been added.

`scipy.optimize.curve_fit` now has more controllable error estimation via the
``absolute_sigma`` keyword.

Support for passing custom minimization methods to ``optimize.minimize()``
and ``optimize.minimize_scalar()`` has been added, currently useful especially
for combining ``optimize.basinhopping()`` with custom local optimizer routines.


``scipy.stats`` improvements
----------------------------

A new class `scipy.stats.multivariate_normal` with functionality for
multivariate normal random variables has been added.

A lot of work on the ``scipy.stats`` distribution framework has been done.
Moment calculations (skew and kurtosis mainly) are fixed and verified, all
examples are now runnable, and many small accuracy and performance improvements
for individual distributions were merged.

The new function `scipy.stats.anderson_ksamp` computes the k-sample
Anderson-Darling test for the null hypothesis that k samples come from
the same parent population.


``scipy.signal`` improvements
-----------------------------

``scipy.signal.iirfilter`` and related functions to design Butterworth,
Chebyshev, elliptical and Bessel IIR filters now all use pole-zero ("zpk")
format internally instead of using transformations to numerator/denominator
format.  The accuracy of the produced filters, especially high-order ones, is
improved significantly as a result.

The new function `scipy.signal.vectorstrength` computes the vector strength,
a measure of phase synchrony, of a set of events.


``scipy.special`` improvements
------------------------------

The functions `scipy.special.boxcox` and `scipy.special.boxcox1p`, which
compute the Box-Cox transformation, have been added.


``scipy.sparse`` improvements
-----------------------------

- Significant performance improvement in CSR, CSC, and DOK indexing speed.
- When using Numpy >= 1.9 (to be released in MM 2014), sparse matrices function
  correctly when given to arguments of ``np.dot``, ``np.multiply`` and other
  ufuncs.  With earlier Numpy and Scipy versions, the results of such
  operations are undefined and usually unexpected.
- Sparse matrices are no longer limited to ``2^31`` nonzero elements.  They
  automatically switch to using 64-bit index data type for matrices containing
  more elements.  User code written assuming the sparse matrices use int32 as
  the index data type will continue to work, except for such large matrices.
  Code dealing with larger matrices needs to accept either int32 or int64
  indices.


Deprecated features
===================

``anneal``
----------

The global minimization function `scipy.optimize.anneal` is deprecated.
All users should use the `scipy.optimize.basinhopping` function instead.

``scipy.stats``
---------------

``randwcdf`` and ``randwppf`` functions are deprecated. All users should use
distribution-specific ``rvs`` methods instead.

Probability calculation aliases ``zprob``, ``fprob`` and ``ksprob`` are
deprecated. Use instead the ``sf`` methods of the corresponding distributions
or the ``special`` functions directly.

``scipy.interpolate``
---------------------

``PiecewisePolynomial`` class is deprecated.


Backwards incompatible changes
==============================

scipy.special.lpmn
------------------

``lpmn`` no longer accepts complex-valued arguments. A new function
``clpmn`` with uniform complex analytic behavior has been added, and
it should be used instead.

scipy.sparse.linalg
-------------------

Eigenvectors in the case of generalized eigenvalue problem are normalized to
unit vectors in 2-norm, rather than following the LAPACK normalization
convention.

The deprecated UMFPACK wrapper in ``scipy.sparse.linalg`` has been removed due
to license and install issues.  If available, ``scikits.umfpack`` is still used
transparently in the ``spsolve`` and ``factorized`` functions.  Otherwise,
SuperLU is used instead in these functions.

scipy.stats
-----------

The deprecated functions ``glm``, ``oneway`` and ``cmedian`` have been removed
from ``scipy.stats``.

``stats.scoreatpercentile`` now returns an array instead of a list of
percentiles.

scipy.interpolate
-----------------

The API for computing derivatives of a monotone piecewise interpolation has
changed: if `p` is a ``PchipInterpolator`` object, `p.derivative(der)`
returns a callable object representing the derivative of `p`. For in-place
derivatives use the second argument of the `__call__` method:
`p(0.1, der=2)` evaluates the second derivative of `p` at `x=0.1`.

The method `p.derivatives` has been removed.

SciPy 0.13.3 Release Notes

SciPy 0.13.3 is a bug-fix release with no new features compared to 0.13.2. Both the weave and the ndimage.label bugs were severe regressions in 0.13.0, hence this release.
Issues fixed

    3148: fix a memory leak in ndimage.label.
    3216: fix weave issue with too long file names for MSVC.

Other changes

    Update Sphinx theme used for html docs so >>> in examples can be toggled.

SciPy 0.13.2 Release Notes

SciPy 0.13.2 is a bug-fix release with no new features compared to 0.13.1.
Issues fixed

    3096: require Cython 0.19, earlier versions have memory leaks in fused types
    3079: ndimage.label fix swapped 64-bitness test
    3108: optimize.fmin_slsqp constraint violation

SciPy 0.13.1 Release Notes

SciPy 0.13.1 is a bug-fix release with no new features compared to 0.13.0. The only changes are several fixes in ndimage, one of which was a serious regression in ndimage.label (Github issue 3025), which gave incorrect results in 0.13.0.
Issues fixed

    3025: ndimage.label returns incorrect results in scipy 0.13.0
    1992: ndimage.label return type changed from int32 to uint32
    1992: ndimage.find_objects doesn't work with int32 input in some cases

SciPy 0.13.0 Release Notes
==========================

.. contents::

SciPy 0.13.0 is the culmination of 7 months of hard work. It contains
many new features, numerous bug-fixes, improved test coverage and
better documentation.  There have been a number of deprecations and
API changes in this release, which are documented below.  All users
are encouraged to upgrade to this release, as there are a large number
of bug-fixes and optimizations.  Moreover, our development attention
will now shift to bug-fix releases on the 0.13.x branch, and on adding
new features on the master branch.

This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater.
Highlights of this release are:

  - support for fancy indexing and boolean comparisons with sparse matrices
  - interpolative decompositions and matrix functions in the linalg module
  - two new trust-region solvers for unconstrained minimization


New features
============

``scipy.integrate`` improvements
--------------------------------

N-dimensional numerical integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A new function `scipy.integrate.nquad`, which provides N-dimensional
integration functionality with a more flexible interface than ``dblquad`` and
``tplquad``, has been added.

``dopri*`` improvements
^^^^^^^^^^^^^^^^^^^^^^^

The intermediate results from the ``dopri`` family of ODE solvers can now be
accessed by a *solout* callback function.


``scipy.linalg`` improvements
-----------------------------

Interpolative decompositions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Scipy now includes a new module `scipy.linalg.interpolative`
containing routines for computing interpolative matrix decompositions
(ID). This feature is based on the ID software package by
P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously
adapted for Python in the PymatrixId package by K.L. Ho.

Polar decomposition
^^^^^^^^^^^^^^^^^^^

A new function `scipy.linalg.polar`, to compute the polar decomposition
of a matrix, was added.

BLAS level 3 functions
^^^^^^^^^^^^^^^^^^^^^^

The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and
``her2k`` are now wrapped in `scipy.linalg`.

Matrix functions
^^^^^^^^^^^^^^^^

Several matrix function algorithms have been implemented or updated following
detailed descriptions in recent papers of Nick Higham and his co-authors.
These include the matrix square root (``sqrtm``), the matrix logarithm
(``logm``), the matrix exponential (``expm``) and its Frechet derivative
(``expm_frechet``), and fractional matrix powers (``fractional_matrix_power``).


``scipy.optimize`` improvements
-------------------------------

Trust-region unconstrained minimization algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``minimize`` function gained two trust-region solvers for unconstrained
minimization: ``dogleg`` and ``trust-ncg``.


``scipy.sparse`` improvements
-----------------------------

Boolean comparisons and sparse matrices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

All sparse matrix types now support boolean data, and boolean operations.  Two
sparse matrices `A` and `B` can be compared in all the expected ways `A < B`,
`A >= B`, `A != B`, producing similar results as dense Numpy arrays.
Comparisons with dense matrices and scalars are also supported.

CSR and CSC fancy indexing
^^^^^^^^^^^^^^^^^^^^^^^^^^

Compressed sparse row and column sparse matrix types now support fancy indexing
with boolean matrices, slices, and lists. So where A is a (CSC or CSR) sparse
matrix, you can do things like::

    >>> A[A > 0.5] = 1  # since Boolean sparse matrices work
    >>> A[:2, :3] = 2
    >>> A[[1,2], 2] = 3


``scipy.sparse.linalg`` improvements
------------------------------------

The new function ``onenormest`` provides a lower bound of the 1-norm of a
linear operator and has been implemented according to Higham and Tisseur
(2000).  This function is not only useful for sparse matrices, but can also be
used to estimate the norm of products or powers of dense matrices without
explictly building the intermediate matrix.

The multiplicative action of the matrix exponential of a linear operator
(``expm_multiply``) has been implemented following the description in Al-Mohy
and Higham (2011).

Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can now be
multiplied, added to each other, and exponentiated, producing new linear
operators. This enables easier construction of composite linear operations.


``scipy.spatial`` improvements
------------------------------

The vertices of a `ConvexHull` can now be accessed via the `vertices` attribute,
which gives proper orientation in 2-D.


``scipy.signal`` improvements
-----------------------------

The cosine window function `scipy.signal.cosine` was added.


``scipy.special`` improvements
------------------------------

New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were added.
These functions can simplify and speed up code that has to calculate
``x * log(y)`` and give 0 when ``x == 0``.


``scipy.io`` improvements
-------------------------

Unformatted Fortran file reader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The new class `scipy.io.FortranFile` facilitates reading unformatted
sequential files written by Fortran code.

``scipy.io.wavfile`` enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.io.wavfile.write` now accepts a file buffer. Previously it only
accepted a filename.

`scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle floating
point WAV files.


``scipy.interpolate`` improvements
----------------------------------

B-spline derivatives and antiderivatives
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`scipy.interpolate.splder` and `scipy.interpolate.splantider` functions
for computing B-splines that represent derivatives and antiderivatives
of B-splines were added.  These functions are also available in the
class-based FITPACK interface as ``UnivariateSpline.derivative`` and
``UnivariateSpline.antiderivative``.


``scipy.stats`` improvements
----------------------------

Distributions now allow using keyword parameters in addition to
positional parameters in all methods.

The function `scipy.stats.power_divergence` has been added for the
Cressie-Read power divergence statistic and goodness of fit test.
Included in this family of statistics is the "G-test"
(http://en.wikipedia.org/wiki/G-test).

`scipy.stats.mood` now accepts multidimensional input.

An option was added to `scipy.stats.wilcoxon` for continuity correction.

`scipy.stats.chisquare` now has an `axis` argument.

`scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments.


Deprecated features
===================

``expm2`` and ``expm3``
-----------------------

The matrix exponential functions `scipy.linalg.expm2` and `scipy.linalg.expm3`
are deprecated. All users should use the numerically more robust
`scipy.linalg.expm` function instead.

``scipy.stats`` functions
-------------------------

`scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used
instead.

`scipy.stats.glm` is deprecated.  `scipy.stats.ttest_ind` is an equivalent
function; more full-featured general (and generalized) linear model
implementations can be found in statsmodels.

`scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used instead.


Backwards incompatible changes
==============================

LIL matrix assignment
---------------------
Assigning values to LIL matrices with two index arrays now works similarly as
assigning into ndarrays::

    >>> x = lil_matrix((3, 3))
    >>> x[[0,1,2],[0,1,2]]=[0,1,2]
    >>> x.todense()
    matrix([[ 0.,  0.,  0.],
            [ 0.,  1.,  0.],
            [ 0.,  0.,  2.]])

rather than giving the result::

    >>> x.todense()
    matrix([[ 0.,  1.,  2.],
            [ 0.,  1.,  2.],
            [ 0.,  1.,  2.]])

Users relying on the previous behavior will need to revisit their code.
The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = ...`.


Deprecated ``radon`` function removed
-------------------------------------

The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has been
removed.  Users can find a more full-featured ``radon`` function in
scikit-image.


Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions``
--------------------------------------------------------------------------

The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have
been removed from the distributions in ``scipy.stats``.

Changes to MATLAB file readers / writers
----------------------------------------

The major change is that 1D arrays in numpy now become row vectors (shape 1, N)
when saved to a MATLAB 5 format file.  Previously 1D arrays saved as column
vectors (N, 1).  This is to harmonize the behavior of writing MATLAB 4 and 5
formats, and adapt to the defaults of numpy and MATLAB - for example
``np.atleast_2d`` returns 1D arrays as row vectors.

Trying to save arrays of greater than 2 dimensions in MATLAB 4 format now raises
an error instead of silently reshaping the array as 2D.

``scipy.io.loadmat('afile')`` used to look for `afile` on the Python system path
(``sys.path``); now ``loadmat`` only looks in the current directory for a
relative path filename.


Other changes
=============

Security fix: ``scipy.weave`` previously used temporary directories in an
insecure manner under certain circumstances.

Cython is now required to build *unreleased* versions of scipy.
The C files generated from Cython sources are not included in the git repo
anymore.  They are however still shipped in source releases.

The code base received a fairly large PEP8 cleanup.  A ``tox pep8``
command has been added; new code should pass this test command.

Scipy cannot be compiled with gfortran 4.1 anymore (at least on RH5), likely
due to that compiler version not supporting entry constructs well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants