Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update numpy to 1.15.0 #58

Merged
merged 1 commit into from Aug 6, 2018
Merged

Conversation

pyup-bot
Copy link
Contributor

@pyup-bot pyup-bot commented Aug 6, 2018

This PR updates numpy from 1.14.3 to 1.15.0.

Changelog

1.15.0

==========================


Highlights
==========

* NumPy has switched to pytest for testing.


New functions
=============

* `np.gcd` and `np.lcm`, to compute the greatest common divisor and least
common multiple.
* `np.ma.stack`, the `np.stack` array-joining function generalized to masked
arrays.
* ``quantile`` function, an interface to ``percentile`` without factors of 100
* ``nanquantile`` function, an interface to ``nanpercentile`` without factors
of 100

* `np.printoptions`, a context manager that sets print options temporarily
for the scope of the ``with`` block::

 >>> with np.printoptions(precision=2):
 ...     print(np.array([2.0]) / 3)
 [0.67]

* `np.histogram_bin_edges`, a function to get the edges of the bins used by a histogram
without needing to calculate the histogram.

* `npy_get_floatstatus_barrier`` and ``npy_clear_floatstatus_barrier`` have been added to
deal with compiler optimization changing the order of operations. See below for details.

Deprecations
============

* Aliases of builtin `pickle` functions are deprecated, in favor of their
unaliased ``pickle.<func>`` names:

* `np.loads`
* `np.core.numeric.load`
* `np.core.numeric.loads`
* `np.ma.loads`, `np.ma.dumps`
* `np.ma.load`, `np.ma.dump` - these functions already failed on python 3,
 when called with a string.

* Multidimensional indexing with anything but a tuple is
deprecated. This means that code such as ``ind = [slice(None), 0]``,
``arr[[slice(None), 0]]`` should be changed to ``arr[tuple(ind)]``. This is
necessary to avoid ambiguity in expressions such as ``arr[[[0, 1], [0, 1]]]``
which currently is interpreted as ``arr[array([0, 1]), array([0, 1])]``.
In future, this will be interpreted as ``arr[array([[0, 1], [0, 1]])]``.

* Direct imports from the following modules is deprecated. All testing related
imports should come from `numpy.testing`.
* `np.testing.utils`
* `np.testing.decorators`
* `np.testing.nosetester`
* `np.testing.noseclasses`
* `np.core.umath_tests`

* Giving a generator to `np.sum` is now deprecated. This was undocumented, but
worked. Previously, it would calculate the sum of the generator expression.
In the future, it might return a different result. Use `np.sum(np.from_iter(generator))`
or the built-in Python `sum` instead.

* Users of the C-API should call ``PyArrayResolveWriteBackIfCopy`` or
``PyArray_DiscardWritbackIfCopy`` on any array with the ``WRITEBACKIFCOPY``
flag set, before the array is deallocated. A deprecation warning will be
emitted if those calls are not used when needed.

* Users of ``nditer`` should use the nditer object as a context manager
anytime one of the iterator operands is writeable, so that numpy can
manage writeback semantics, or should call ``it.close()``. A
`RuntimeWarning` may be emitted otherwise in these cases.

* The ``normed`` argument of ``np.histogram``, deprecated long ago in 1.6.0,
now emits a ``DeprecationWarning``.


Future Changes
==============


Compatibility notes
===================

The ``NpzFile`` returned by ``np.savez`` is now a `collections.abc.Mapping`
---------------------------------------------------------------------------
This means it behaves like a readonly dictionary, and has a new ``.values()``
method and ``len()`` implementation.

On python 3, this means that ``.iteritems()``, ``.iterkeys()`` have been
deprecated, and ``.keys()`` and ``.items()`` now return views and not lists.
This is consistent with how the builtin ``dict`` type changed between python 2
and python 3.

Under certain conditions, nditer must be used in a context manager
------------------------------------------------------------------
When using an nditer with the ``"writeonly"`` or ``"readwrite"`` flags, there
are some circumstances where nditer doesn't actually give you a view onto the
writable array. Instead, it gives you a copy, and if you make changes to the
copy, nditer later writes those changes back into your actual array. Currently,
this writeback occurs when the array objects are garbage collected, which makes
this API error-prone on CPython and entirely broken on PyPy. Therefore,
``nditer`` should now be used as a context manager whenever using ``nditer``
with writeable arrays (``with np.nditer(...) as it: ...``). You may also
explicitly call ``it.close()`` for cases where a context manager is unusable,
for instance in generator expressions.

Numpy has switched to using pytest instead of nose for testing
--------------------------------------------------------------
The last nose release was 1.3.7 in June, 2015, and development of that tool has
ended, consequently NumPy has now switched to using pytest. The old decorators
and nose tools that were previously used by some downstream projects remain
available, but will not be maintained. The standard testing utilities,
`assert_almost_equal` and such, are not be affected by this change except for
the nose specific functions `import_nose` and `raises`. Those functions are
not used in numpy, but are kept for downstream compatibility.

Numpy no longer monkey-patches ``ctypes`` with ``__array_interface__``
----------------------------------------------------------------------
Previously numpy added ``__array_interface__`` attributes to all the integer
types from ``ctypes``.

``np.ma.notmasked_contiguous`` and ``np.ma.flatnotmasked_contiguous`` always return lists
-----------------------------------------------------------------------------------------
This was always the documented behavior, but in reality the result used to be
any of slice, None, or list.

All downstream users seem to use detect the `None` result from
``flatnotmasked_contiguous`` and replace it with ``[]``.
These callers will continue to work as before.

``np.squeeze`` now respects the API expectation of objects that do not handle an ``axis`` argument
--------------------------------------------------------------------------------------------------
Prior to version ``1.7.0`` ``np.squeeze`` did not have an ``axis`` argument and all empty axes were removed
by default. After incorporation of an ``axis`` argument, it was possible to selectively squeeze single
or multiple empty axes, but the old API expectation was not respected because the axes could still be
selectively removed (silent success) in an object depending on the old API. The silent success is no
longer possible, and objects expecting the old API are respected. The silent success was prevented
by removing the interception of an otherwise-normal Exception when ``axis`` was provided to an object
using the old API.

unstructured void array's ``.item`` method now returns a bytes object
---------------------------------------------------------------------
``.item`` now returns a ``bytes`` object instead of a buffer or byte array.
This may affect code which assumed the return value was mutable, which is no
longer the case.

``copy.copy`` and ``copy.deepcopy`` no longer turn ``masked`` into an array
----------------------------------------------------------------------------
Since ``np.ma.masked`` is a readonly scalar, copying should be a no-op. These
functions now behave consistently with ``np.copy()``.


Multifield Indexing of Structured Arrays will still return a copy
-----------------------------------------------------------------
The change that multi-field indexing of structured arrays returns a view
instead of a copy is pushed back to 1.16. A new method
``numpy.lib.recfunctions.repack_fields`` has been introduced to help mitigate
the effects of this change, which can be used to write code compatible with
both numpy 1.15 and 1.16. For more information on how to update code to account
for this future change see "basics/structured arrays/accessing multiple fields"
in the user guide.

C API changes
=============

* Functions ``npy_get_floatstatus_barrier`` and ``npy_clear_floatstatus_barrier``
have been added and should be used in place of the ``npy_get_floatstatus``and
``npy_clear_status`` functions. Optimizing compilers like GCC 8.1 and Clang
were rearranging the order of operations when the previous functions were
used in the ufunc SIMD functions, resulting in the floatstatus flags being '
checked before the operation whose status we wanted to check was run.
See `10339 <https://github.com/numpy/numpy/issues/10370>`__.

* ``PyArray_GetDTypeTransferFunction`` now defaults to using user-defined
``copyswapn`` / ``copyswap`` for user-defined dtypes. If this causes a
significant performance hit, consider implementing ``copyswapn`` to reflect
the implementation of ``PyArray_GetStridedCopyFn``.
See `10898 <https://github.com/numpy/numpy/pull/10898>`__.

New Features
============

``np.gcd`` and ``np.lcm`` ufuncs added for integer and objects types
--------------------------------------------------------------------
These compute the greatest common divisor, and lowest common multiple,
respectively. These work on all the numpy integer types, as well as the
builtin arbitrary-precision `Decimal` and `long` types.

Support for cross-platform builds for iOS
-----------------------------------------
The build system has been modified to add support for the
``_PYTHON_HOST_PLATFORM`` environment variable, used by ``distutils`` when
compiling on one platform for another platform. This makes it possible to
compile NumPy for iOS targets.

This only enables you to compile NumPy for one specific platform at a time.
Creating a full iOS-compatible NumPy package requires building for the 5
architectures supported by iOS (i386, x86_64, armv7, armv7s and arm64), and
combining these 5 compiled builds products into a single "fat" binary.

``return_indices`` keyword added for ``np.intersect1d``
-------------------------------------------------------
New keyword ``return_indices`` returns the indices of the two input arrays
that correspond to the common elements.

``np.quantile`` and ``np.nanquantile``
--------------------------------------
Like ``np.percentile`` and ``np.nanpercentile``, but takes quantiles in [0, 1]
rather than percentiles in [0, 100]. ``np.percentile`` is now a thin wrapper
around ``np.quantile`` with the extra step of dividing by 100.


Build system
------------
Added experimental support for the 64-bit RISC-V architecture.


Improvements
============

``np.ufunc.reduce`` and related functions now accept an initial value
---------------------------------------------------------------------
``np.ufunc.reduce``, ``np.sum``, ``np.prod``, ``np.min`` and ``np.max`` all
now accept an ``initial`` keyword argument that specifies the value to start
the reduction with.

``np.flip`` can operate over multiple axes
------------------------------------------
``np.flip`` now accepts None, or tuples of int, in its ``axis`` argument. If
axis is None, it will flip over all the axes.

``histogram`` and ``histogramdd`` functions have moved to ``np.lib.histograms``
------------------------------------------------------------------------------
These were originally found in ``np.lib.function_base``. They are still
available under their un-scoped ``np.histogram(dd)`` names, and
to maintain compatibility, aliased at ``np.lib.function_base.histogram(dd)``.

Code that does ``from np.lib.function_base import *`` will need to be updated
with the new location, and should consider not using ``import *`` in future.

``histogram`` will accept NaN values when explicit bins are given
-----------------------------------------------------------------
Previously it would fail when trying to compute a finite range for the data.
Since the range is ignored anyway when the bins are given explicitly, this error
was needless.

Note that calling `histogram` on NaN values continues to raise the
`RuntimeWarning`s typical of working with nan values, which can be silenced
as usual with `errstate`.

``histogram`` works on datetime types, when explicit bin edges are given
------------------------------------------------------------------------
Dates, times, and timedeltas can now be histogrammed. The bin edges must be
passed explicitly, and are not yet computed automatically.

``histogram`` "auto" estimator handles limited variance better
------------------------------------------------------------------------
No longer does an IQR of 0 result in `n_bins=1`, rather the number of bins
chosen is related to the data size in this situation.

``histogram`` and ``histogramdd`` return edges matching the float type of the data
----------------------------------------------------------------------------------
When passed ``float16``, ``np.float32``, or ``np.longdouble`` data, the
returned edges are now of the same dtype. Previously, ``histogram`` would only
return the same type if explicit bins were given, and ``histogram`` would
produce ``float64`` bins no matter what the inputs.

``histogramdd`` allows explicit ranges to be given in a subset of axes
----------------------------------------------------------------------
The ``range`` argument of `histogramdd` can now contain ``None`` values to
indicate that the range for the corresponding axis should be computed from the
data. Previously, this could not be specified on a per-axis basis.

The normed arguments of ``histogramdd`` and ``histogram2d`` have been renamed
-----------------------------------------------------------------------------
These arguments are now called ``density``, which is consistent with
``histogram``. The old argument continues to work, but the new name should be
preferred.

``np.r_`` works with 0d arrays, and ``np.ma.mr_`` works with ``np.ma.masked``
----------------------------------------------------------------------------
0d arrays passed to the `r_` and `mr_` concatenation helpers are now treated as
though they are arrays of length 1. Previously, passing these was an error.
As a result, ``np.ma.mr_`` now works correctly on the ``masked`` constant.

``np.ptp`` accepts a ``keepdims`` argument, and extended axis tuples
--------------------------------------------------------------------
``np.ptp`` (peak-to-peak) can now work over multiple axes, just like `max` and
`min`.

``MaskedArray.astype`` now is identical to ``ndarray.astype``
-------------------------------------------------------------
This means it takes all the same arguments, making more code written for
ndarray work for masked array too.

Enable AVX2/AVX512 at compile time
-------------------------------------------------------------
Change to simd.inc.src to use AVX2 or AVX512 at compile time. Solving the gap
that if compile numpy for avx2 (or 512) with -march=native, still get the SSE
code for the simd functions even though rest of the code gets AVX2.

``nan_to_num`` always returns scalars when receiving scalar or 0d inputs
------------------------------------------------------------------------
Previously an array was returned for integer scalar inputs, which is
inconsistent with the behavior for float inputs, and that of ufuncs in general.
For all types of scalar or 0d input, the result is now a scalar.

``np.flatnonzero`` works on numpy-convertible types
---------------------------------------------------
``np.flatnonzero`` now uses ``np.ravel(a)`` instead of ``a.ravel()``, so it
works for lists, tuples, etc.

``np.interp`` returns numpy scalars rather than builtin scalars
---------------------------------------------------------------
Previously ``np.interp(0.5, [0, 1], [10, 20])`` would return a ``float``, but
now it returns a ``np.float64`` object, which more closely matches the behavior
of other functions.

Additionally, the special case of ``np.interp(object_array_0d, ...)`` is no
longer supported, as ``np.interp(object_array_nd)`` was never supported anyway.

As a result of this change, the ``period`` argument can now be used on 0d
arrays.

Allow dtype field names to be unicode in Python 2
---------------------------------------------------------------
Previously ``np.dtype([(u'name', float)])`` would raise a ``TypeError`` in
Python 2, as only bytestrings were allowed in field names. Now any unicode
string field names will be encoded with the ``ascii`` codec, raising a
``UnicodeEncodeError`` upon failure.

This change makes it easier to write Python 2/3 compatible code using
``from __future__ import unicode_literals``, which previously would cause
string literal field names to raise a TypeError in Python 2.

Comparison ufuncs accept ``dtype=object``, overriding the default ``bool``
--------------------------------------------------------------------------
This allows object arrays of symbolic types, which override ``==`` and other
operators to return expressions, to be compared elementwise with
``np.equal(a, b, dtype=object)``.

``sort`` functions accept ``kind='stable'``
-------------------------------------------
Up until now, to perform a stable sort on the data, the user must do:

 >>> np.sort([5, 2, 6, 2, 1], kind='mergesort')
 [1, 2, 2, 5, 6]

because merge sort is the only stable sorting algorithm available in
NumPy. However, having kind='mergesort' does not make it explicit that
the user wants to perform a stable sort thus harming the readability.

This change allows the user to specify kind='stable' thus clarifying
the intent.

Do not make temporary copies for in-place accumulation
------------------------------------------------------
When ufuncs perform accumulation they no longer make temporary copies because
of the overlap between input an output, that is, the next element accumulated
is added before the accumulated result is stored in its place, hence the
overlap is safe. Avoiding the copy results in faster execution.

``linalg.matrix_power`` can now handle stacks of matrices
---------------------------------------------------------
Like other functions in ``linalg``, ``matrix_power`` can now deal with arrays
of dimension larger than 2, which are treated as stacks of matrices. As part
of the change, to further improve consistency, the name of the first argument
has been changed to ``a`` (from ``M``), and the exceptions for non-square
matrices have been changed to ``LinAlgError`` (from ``ValueError``).

Increased performance in ``random.permutation`` for multidimensional arrays
---------------------------------------------------------------------------
``permutation`` uses the fast path in ``random.shuffle`` for all input
array dimensions.  Previously the fast path was only used for 1-d arrays.

Generalized ufuncs now accept ``axes``, ``axis`` and ``keepdims`` arguments
---------------------------------------------------------------------------
One can control over which axes a generalized ufunc operates by passing in an
``axes`` argument, a list of tuples with indices of particular axes.  For
instance, for a signature of ``(i,j),(j,k)->(i,k)`` appropriate for matrix
multiplication, the base elements are two-dimensional matrices and these are
taken to be stored in the two last axes of each argument.  The corresponding
axes keyword would be ``[(-2, -1), (-2, -1), (-2, -1)]``. If one wanted to
use leading dimensions instead, one would pass in ``[(0, 1), (0, 1), (0, 1)]``.

For simplicity, for generalized ufuncs that operate on 1-dimensional arrays
(vectors), a single integer is accepted instead of a single-element tuple, and
for generalized ufuncs for which all outputs are scalars, the (empty) output
tuples can be omitted.  Hence, for a signature of ``(i),(i)->()`` appropriate
for an inner product, one could pass in ``axes=[0, 0]`` to indicate that the
vectors are stored in the first dimensions of the two inputs arguments.

As a short-cut for generalized ufuncs that are similar to reductions, i.e.,
that act on a single, shared core dimension such as the inner product example
above, one can pass an ``axis`` argument. This is equivalent to passing in
``axes`` with identical entries for all arguments with that core dimension
(e.g., for the example above, ``axes=[(axis,), (axis,)]``).

Furthermore, like for reductions, for generalized ufuncs that have inputs that
all have the same number of core dimensions and outputs with no core dimension,
one can pass in ``keepdims`` to leave a dimension with size 1 in the outputs,
thus allowing proper broadcasting against the original inputs. The location of
the extra dimension can be controlled with ``axes``. For instance, for the
inner-product example, ``keepdims=True, axes=[-2, -2, -2]`` would act on the
inner-product example, ``keepdims=True, axis=-2`` would act on the
one-but-last dimension of the input arguments, and leave a size 1 dimension in
that place in the output.

float128 values now print correctly on ppc systems
--------------------------------------------------
Previously printing float128 values was buggy on ppc, since the special
double-double floating-point-format on these systems was not accounted for.
float128s now print with correct rounding and uniqueness.

Warning to ppc users: You should upgrade glibc if it is version <=2.23,
especially if using float128. On ppc, glibc's malloc in these version often
misaligns allocated memory which can crash numpy when using float128 values.

New ``np.take_along_axis`` and ``np.put_along_axis`` functions
--------------------------------------------------------------
When used on multidimensional arrays, ``argsort``, ``argmin``, ``argmax``, and
``argpartition`` return arrays that are difficult to use as indices.
``take_along_axis`` provides an easy way to use these indices to lookup values
within an array, so that::

 np.take_along_axis(a, np.argsort(a, axis=axis), axis=axis)

is the same as::

 np.sort(a, axis=axis)

``np.put_along_axis`` acts as the dual operation for writing to these indices
within an array.

.. note:: Implementations of ``__array_ufunc__`` should ensure that they can
       handle either ``axis`` or ``axes``.  In future, we may convert
       ``axis`` to ``axes`` before passing it on.

Changes
=======


==========================

1.14.5

==========================

This is a bugfix release for bugs reported following the 1.14.4 release. The
most significant fixes are:

* fixes for compilation errors on alpine and NetBSD

The Python versions supported in this release are 2.7 and 3.4 - 3.6. The Python
3.6 wheels available from PIP are built with Python 3.6.2 and should be
compatible with all previous versions of Python 3.6. The source releases were
cythonized with Cython 0.28.2 and should work for the upcoming Python 3.7.

Contributors
============

A total of 1 person contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Charles Harris

Pull requests merged
====================

A total of 2 pull requests were merged for this release.

* `11274 <https://github.com/numpy/numpy/pull/11274>`__: BUG: Correct use of NPY_UNUSED.
* `11294 <https://github.com/numpy/numpy/pull/11294>`__: BUG: Remove extra trailing parentheses.



=========================

1.14.4

==========================

This is a bugfix release for bugs reported following the 1.14.3 release. The
most significant fixes are:

* fixes for compiler instruction reordering that resulted in NaN's not being
properly propagated in `np.max` and `np.min`,

* fixes for bus faults on SPARC and older ARM due to incorrect alignment
checks.

There are also improvements to printing of long doubles on PPC platforms. All
is not yet perfect on that platform, the whitespace padding is still incorrect
and is to be fixed in numpy 1.15, consequently NumPy still fails some
printing-related (and other) unit tests on ppc systems. However, the printed
values are now correct.

Note that NumPy will error on import if it detects incorrect float32 `dot`
results. This problem has been seen on the Mac when working in the Anaconda
enviroment and is due to a subtle interaction between MKL and PyQt5.  It is not
strictly a NumPy problem, but it is best that users be aware of it.  See the
gh-8577 NumPy issue for more information.

The Python versions supported in this release are 2.7 and 3.4 - 3.6. The Python
3.6 wheels available from PIP are built with Python 3.6.2 and should be
compatible with all previous versions of Python 3.6. The source releases were
cythonized with Cython 0.28.2 and should work for the upcoming Python 3.7.

Contributors
============

A total of 7 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Allan Haldane
* Charles Harris
* Marten van Kerkwijk
* Matti Picus
* Pauli Virtanen
* Ryan Soklaski +
* Sebastian Berg

Pull requests merged
====================

A total of 11 pull requests were merged for this release.

* `11104 <https://github.com/numpy/numpy/pull/11104>`__: BUG: str of DOUBLE_DOUBLE format wrong on ppc64
* `11170 <https://github.com/numpy/numpy/pull/11170>`__: TST: linalg: add regression test for gh-8577
* `11174 <https://github.com/numpy/numpy/pull/11174>`__: MAINT: add sanity-checks to be run at import time
* `11181 <https://github.com/numpy/numpy/pull/11181>`__: BUG: void dtype setup checked offset not actual pointer for alignment
* `11194 <https://github.com/numpy/numpy/pull/11194>`__: BUG: Python2 doubles don't print correctly in interactive shell.
* `11198 <https://github.com/numpy/numpy/pull/11198>`__: BUG: optimizing compilers can reorder call to npy_get_floatstatus
* `11199 <https://github.com/numpy/numpy/pull/11199>`__: BUG: reduce using SSE only warns if inside SSE loop
* `11203 <https://github.com/numpy/numpy/pull/11203>`__: BUG: Bytes delimiter/comments in genfromtxt should be decoded
* `11211 <https://github.com/numpy/numpy/pull/11211>`__: BUG: Fix reference count/memory leak exposed by better testing
* `11219 <https://github.com/numpy/numpy/pull/11219>`__: BUG: Fixes einsum broadcasting bug when optimize=True
* `11251 <https://github.com/numpy/numpy/pull/11251>`__: DOC: Document 1.14.4 release.


=========================
Links

@coveralls
Copy link

Pull Request Test Coverage Report for Build 183

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 77.7%

Totals Coverage Status
Change from base Build 174: 0.0%
Covered Lines: 777
Relevant Lines: 1000

💛 - Coveralls

@mhalagan mhalagan merged commit 2439bf4 into master Aug 6, 2018
@mhalagan mhalagan deleted the pyup-update-numpy-1.14.3-to-1.15.0 branch August 6, 2018 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants