Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Remove maskna API from ndarray, and all (and only) the code supportin…

…g it

The original masked-NA-NEP branch contained a large number of changes
in addition to the core NA support. For example:
 - ufunc.__call__ support for where= argument
 - nditer support for arbitrary masks (in support of where=)
 - ufunc.reduce support for simultaneous reduction over multiple axes
 - a new "array assignment API"
 - ndarray.diagonal() returning a view in all cases
 - bug-fixes in __array_priority__ handling
 - datetime test changes
etc. There's no consensus yet on what should be done with the
maskna-related part of this branch, but the rest is generally useful
and uncontroversial, so the goal of this branch is to identify exactly
which code changes are involved in maskna support.

The basic strategy used to create this patch was:
 - Remove the new masking-related fields from ndarray, so no arrays
   are masked
 - Go through and remove all the code that this makes
   dead/inaccessible/irrelevant, in a largely mechanical fashion. So
   for example, if I saw 'if (PyArray_HASMASK(a)) { ... }' then that
   whole block was obviously just dead code if no arrays have masks,
   and I removed it. Likewise for function arguments like skipna that
   are useless if there aren't any NAs to skip.

This changed the signature of a number of functions that were newly
exposed in the numpy public API. I've removed all such functions from
the public API, since releasing them with the NA-less signature in 1.7
would create pointless compatibility hassles later if and when we add
back the NA-related functionality. Most such functions are removed by
this commit; the exception is PyArray_ReduceWrapper, which requires
more extensive surgery, and will be handled in followup commits.

I also removed the new ndarray.setasflat method. Reason: a comment
noted that the only reason this was added was to allow easier testing
of one branch of PyArray_CopyAsFlat. That branch is now the main
branch, so that isn't an issue. Nonetheless this function is arguably
useful, so perhaps it should have remained, but I judged that since
numpy's API is already hairier than we would like, it's not a good
idea to add extra hair "just in case". (Also AFAICT the test for this
method in test_maskna was actually incorrect, as noted here:
   https://github.com/njsmith/numpyNEP/blob/master/numpyNEP.py
so I'm not confident that it ever worked in master, though I haven't
had a chance to follow-up on this.)

I also removed numpy.count_reduce_items, since without skipna it
became trivial.

I believe that these are the only exceptions to the "remove dead code"
strategy.
  • Loading branch information...
commit b272bc605ce7784be5b3edb13ad7afe22b04e71f 1 parent 1b6582d
@njsmith njsmith authored
Showing with 789 additions and 11,563 deletions.
  1. +0 −306 doc/source/reference/arrays.maskna.rst
  2. +0 −1  doc/source/reference/arrays.rst
  3. +0 −114 doc/source/reference/c-api.array.rst
  4. +0 −592 doc/source/reference/c-api.maskna.rst
  5. +0 −1  doc/source/reference/c-api.rst
  6. +0 −28 doc/source/reference/routines.polynomials.classes.rst
  7. +0 −1  doc/source/reference/routines.rst
  8. +4 −129 numpy/add_newdocs.py
  9. +38 −26 numpy/core/_methods.py
  10. +40 −75 numpy/core/arrayprint.py
  11. +0 −2  numpy/core/code_generators/genapi.py
  12. +8 −29 numpy/core/code_generators/numpy_api.py
  13. +31 −59 numpy/core/fromnumeric.py
  14. +7 −11 numpy/core/function_base.py
  15. +5 −161 numpy/core/include/numpy/ndarraytypes.h
  16. +3 −1 numpy/core/include/numpy/ufuncobject.h
  17. +22 −74 numpy/core/numeric.py
  18. +2 −3 numpy/core/numerictypes.py
  19. +0 −6 numpy/core/setup.py
  20. +9 −5 numpy/core/src/multiarray/array_assign.h
  21. +19 −433 numpy/core/src/multiarray/array_assign_array.c
  22. +18 −239 numpy/core/src/multiarray/array_assign_scalar.c
  23. +9 −83 numpy/core/src/multiarray/arrayobject.c
  24. +0 −33 numpy/core/src/multiarray/arraytypes.c.src
  25. +0 −405 numpy/core/src/multiarray/boolean_ops.c.src
  26. +0 −13 numpy/core/src/multiarray/boolean_ops.h
  27. +0 −4 numpy/core/src/multiarray/buffer.c
  28. +18 −41 numpy/core/src/multiarray/calculation.c
  29. +9 −32 numpy/core/src/multiarray/common.c
  30. +2 −8 numpy/core/src/multiarray/common.h
  31. +1 −65 numpy/core/src/multiarray/conversion_utils.c
  32. +10 −56 numpy/core/src/multiarray/convert.c
  33. +7 −17 numpy/core/src/multiarray/convert_datatype.c
  34. +111 −458 numpy/core/src/multiarray/ctors.c
  35. +0 −16 numpy/core/src/multiarray/ctors.h
  36. +0 −15 numpy/core/src/multiarray/datetime.c
  37. +8 −226 numpy/core/src/multiarray/dtype_transfer.c
  38. +1 −1  numpy/core/src/multiarray/einsum.c.src
  39. +0 −109 numpy/core/src/multiarray/flagsobject.c
  40. +0 −9 numpy/core/src/multiarray/getset.c
  41. +32 −350 numpy/core/src/multiarray/item_selection.c
  42. +2 −3 numpy/core/src/multiarray/item_selection.h
  43. +3 −42 numpy/core/src/multiarray/iterators.c
  44. +2 −6 numpy/core/src/multiarray/iterators.h
  45. +110 −498 numpy/core/src/multiarray/mapping.c
  46. +19 −203 numpy/core/src/multiarray/methods.c
  47. +42 −275 numpy/core/src/multiarray/multiarraymodule.c
  48. +0 −3  numpy/core/src/multiarray/multiarraymodule_onefile.c
  49. +0 −955 numpy/core/src/multiarray/na_mask.c
  50. +0 −58 numpy/core/src/multiarray/na_mask.h
  51. +0 −802 numpy/core/src/multiarray/na_object.c
  52. +0 −60 numpy/core/src/multiarray/na_object.h
  53. +7 −108 numpy/core/src/multiarray/nditer_api.c
  54. +40 −382 numpy/core/src/multiarray/nditer_constr.c
  55. +2 −13 numpy/core/src/multiarray/nditer_impl.h
  56. +20 −116 numpy/core/src/multiarray/nditer_pywrap.c
  57. +33 −715 numpy/core/src/multiarray/reduction.c
  58. +10 −50 numpy/core/src/multiarray/reduction.h
  59. +1 −13 numpy/core/src/multiarray/scalarapi.c
  60. +1 −22 numpy/core/src/multiarray/sequence.c
  61. +13 −136 numpy/core/src/multiarray/shape.c
  62. +8 −51 numpy/core/src/private/lowlevel_strided_loops.h
  63. +43 −324 numpy/core/src/umath/ufunc_object.c
  64. +2 −4 numpy/core/src/umath/ufunc_type_resolution.c
  65. +0 −103 numpy/core/tests/test_api.py
  66. +0 −28 numpy/core/tests/test_indexerrors.py
  67. +0 −1,621 numpy/core/tests/test_maskna.py
  68. +1 −20 numpy/core/tests/test_multiarray.py
  69. +0 −198 numpy/core/tests/test_na.py
  70. +0 −139 numpy/core/tests/test_nditer.py
  71. +0 −58 numpy/core/tests/test_numeric.py
  72. +0 −5 numpy/core/tests/test_regression.py
  73. +2 −10 numpy/lib/function_base.py
  74. +2 −4 numpy/lib/twodim_base.py
  75. +1 −12 numpy/polynomial/chebyshev.py
  76. +1 −12 numpy/polynomial/hermite.py
  77. +1 −12 numpy/polynomial/hermite_e.py
  78. +1 −12 numpy/polynomial/laguerre.py
  79. +1 −12 numpy/polynomial/legendre.py
  80. +5 −24 numpy/polynomial/polynomial.py
  81. +0 −30 numpy/polynomial/tests/test_chebyshev.py
  82. +0 −30 numpy/polynomial/tests/test_hermite.py
  83. +0 −30 numpy/polynomial/tests/test_hermite_e.py
  84. +0 −30 numpy/polynomial/tests/test_laguerre.py
  85. +0 −30 numpy/polynomial/tests/test_legendre.py
  86. +0 −30 numpy/polynomial/tests/test_polynomial.py
  87. +2 −37 numpy/testing/utils.py
View
306 doc/source/reference/arrays.maskna.rst
@@ -1,306 +0,0 @@
-.. currentmodule:: numpy
-
-.. _arrays.maskna:
-
-****************
-NA-Masked Arrays
-****************
-
-.. versionadded:: 1.7.0
-
-NumPy 1.7 adds preliminary support for missing values using an interface
-based on an NA (Not Available) placeholder, implemented as masks in the
-core ndarray. This system is highly flexible, allowing NAs to be used
-with any underlying dtype, and supports creating multiple views of the same
-data with different choices of NAs.
-
-.. note:: The NA API is *experimental*, and may undergo changes in future
- versions of NumPy. The current implementation based on masks will likely be
- supplemented by a second one based on bit-patterns, and it is possible that
- a difference will be made between missing and ignored data.
-
-Other Missing Data Approaches
-=============================
-
-The previous recommended approach for working with missing values was the
-:mod:`numpy.ma` module, a subclass of ndarray written purely in Python.
-By placing NA-masks directly in the NumPy core, it's possible to avoid
-the need for calling "ma.<func>(arr)" instead of "np.<func>(arr)".
-
-Another approach many people have taken is to use NaN as the
-placeholder for missing values. There are a few functions
-like :func:`numpy.nansum` which behave similarly to usage of the
-ufunc.reduce *skipna* parameter.
-
-As experienced in the R language, a programming interface based on an
-NA placeholder is generally more intuitive to work with than direct
-mask manipulation.
-
-Missing Data Model
-==================
-
-The model adopted by NumPy for missing values is that NA is a
-placeholder for a value which is there, but is unknown to computations.
-The value may be temporarily hidden by the mask, or may be unknown
-for any reason, but could be any value the dtype of the array is able
-to hold.
-
-This model affects computations in specific, well-defined ways. Any time
-we have a computation, like *c = NA + 1*, we must reason about whether
-*c* will be an NA or not. The NA is not available now, but maybe a
-measurement will be made later to determine what its value is, so anything
-we calculate must be consistent with it eventually being revealed. One way
-to do this is with thought experiments imagining we have discovered
-the value of this NA. If the NA is 0, then *c* is 1. If the NA is
-100, then *c* is 101. Because the value of *c* is ambiguous, it
-isn't available either, so must be NA as well.
-
-A consequence of separating the NA model from the dtype is that, unlike
-in R, NaNs are not considered to be NA. An NA is a value that is completely
-unknown, whereas a NaN is usually the result of an invalid computation
-as defined in the IEEE 754 floating point arithmetic specification.
-
-Most computations whose input is NA will output NA as well, a property
-known as propagation. Some operations, however, always produce the
-same result no matter what the value of the NA is. The clearest
-example of this is with the logical operations *and* and *or*. Since both
-np.logical_or(True, True) and np.logical_or(False, True) are True,
-all possible boolean values on the left hand side produce the
-same answer. This means that np.logical_or(np.NA, True) can produce
-True instead of the more conservative np.NA. There is a similar case
-for np.logical_and.
-
-A similar, but slightly deceptive, example is wanting to treat (NA * 0.0)
-as 0.0 instead of as NA. This is invalid because the NA might be Inf
-or NaN, in which case the result is NaN instead of 0.0. This idea is
-valid for integer dtypes, but NumPy still chooses to return NA because
-checking this special case would adversely affect performance.
-
-The NA Object
-=============
-
-In the root numpy namespace, there is a new object NA. This is not
-the only possible instance of an NA as is the case for None, since an NA
-may have a dtype associated with it and has been designed for future
-expansion to carry a multi-NA payload. It can be used in computations
-like any value::
-
- >>> np.NA
- NA
- >>> np.NA * 3
- NA(dtype='int64')
- >>> np.sin(np.NA)
- NA(dtype='float64')
-
-To check whether a value is NA, use the :func:`numpy.isna` function::
-
- >>> np.isna(np.NA)
- True
- >>> np.isna(1.5)
- False
- >>> np.isna(np.nan)
- False
- >>> np.isna(np.NA * 3)
- True
- >>> (np.NA * 3) is np.NA
- False
-
-
-Creating NA-Masked Arrays
-=========================
-
-Because having NA support adds some overhead to NumPy arrays, one
-must explicitly request it when creating arrays. There are several ways
-to get an NA-masked array. The easiest way is to include an NA
-value in the list used to construct the array.::
-
- >>> a = np.array([1,3,5])
- >>> a
- array([1, 3, 5])
- >>> a.flags.maskna
- False
-
- >>> b = np.array([1,3,np.NA])
- >>> b
- array([1, 3, NA])
- >>> b.flags.maskna
- True
-
-If one already has an array without an NA-mask, it can be added
-by directly setting the *maskna* flag to True. Assigning an NA
-to an array without NA support will raise an error rather than
-automatically creating an NA-mask, with the idea that supporting
-NA should be an explicit user choice.::
-
- >>> a = np.array([1,3,5])
- >>> a[1] = np.NA
- Traceback (most recent call last):
- File "<stdin>", line 1, in <module>
- ValueError: Cannot assign NA to an array which does not support NAs
- >>> a.flags.maskna = True
- >>> a[1] = np.NA
- >>> a
- array([1, NA, 5])
-
-Most array construction functions have a new parameter *maskna*, which
-can be set to True to produce an array with an NA-mask.::
-
- >>> np.arange(5., maskna=True)
- array([ 0., 1., 2., 3., 4.], maskna=True)
- >>> np.eye(3, maskna=True)
- array([[ 1., 0., 0.],
- [ 0., 1., 0.],
- [ 0., 0., 1.]], maskna=True)
- >>> np.array([1,3,5], maskna=True)
- array([1, 3, 5], maskna=True)
-
-Creating NA-Masked Views
-========================
-
-It will sometimes be desirable to view an array with an NA-mask, without
-adding an NA-mask to that array. This is possible by taking an NA-masked
-view of the array. There are two ways to do this, one which simply
-guarantees that the view has an NA-mask, and another which guarantees that the
-view has its own NA-mask, even if the array already had an NA-mask.
-
-Starting with a non-masked array, we can use the :func:`ndarray.view` method
-to get an NA-masked view.::
-
- >>> a = np.array([1,3,5])
- >>> b = a.view(maskna=True)
-
- >>> b[2] = np.NA
- >>> a
- array([1, 3, 5])
- >>> b
- array([1, 3, NA])
-
- >>> b[0] = 2
- >>> a
- array([2, 3, 5])
- >>> b
- array([2, 3, NA])
-
-
-It is important to be cautious here, though, since if the array already
-has a mask, this will also take a view of that mask. This means the original
-array's mask will be affected by assigning NA to the view.::
-
- >>> a = np.array([1,np.NA,5])
- >>> b = a.view(maskna=True)
-
- >>> b[2] = np.NA
- >>> a
- array([1, NA, NA])
- >>> b
- array([1, NA, NA])
-
- >>> b[1] = 4
- >>> a
- array([1, 4, NA])
- >>> b
- array([1, 4, NA])
-
-
-To guarantee that the view created has its own NA-mask, there is another
-flag *ownmaskna*. Using this flag will cause a copy of the array's mask
-to be created for the view when the array already has a mask.::
-
- >>> a = np.array([1,np.NA,5])
- >>> b = a.view(ownmaskna=True)
-
- >>> b[2] = np.NA
- >>> a
- array([1, NA, 5])
- >>> b
- array([1, NA, NA])
-
- >>> b[1] = 4
- >>> a
- array([1, NA, 5])
- >>> b
- array([1, 4, NA])
-
-
-In general, when an NA-masked view of an array has been taken, any time
-an NA is assigned to an element of the array the data for that element
-will remain untouched. This mechanism allows for multiple temporary
-views with NAs of the same original array.
-
-NA-Masked Reductions
-====================
-
-Many of NumPy's reductions like :func:`numpy.sum` and :func:`numpy.std`
-have been extended to work with NA-masked arrays. A consequence of the
-missing value model is that any NA value in an array will cause the
-output including that value to become NA.::
-
- >>> a = np.array([[1,2,np.NA,3], [0,np.NA,1,1]])
- >>> a.sum(axis=0)
- array([1, NA, NA, 4])
- >>> a.sum(axis=1)
- array([NA, NA], dtype=int64)
-
-This is not always the desired result, so NumPy includes a parameter
-*skipna* which causes the NA values to be skipped during computation.::
-
- >>> a = np.array([[1,2,np.NA,3], [0,np.NA,1,1]])
- >>> a.sum(axis=0, skipna=True)
- array([1, 2, 1, 4])
- >>> a.sum(axis=1, skipna=True)
- array([6, 2])
-
-Iterating Over NA-Masked Arrays
-===============================
-
-The :class:`nditer` object can be used to iterate over arrays with
-NA values just like over normal arrays.::
-
- >>> a = np.array([1,3,np.NA])
- >>> for x in np.nditer(a):
- ... print x,
- ...
- 1 3 NA
- >>> b = np.zeros(3, maskna=True)
- >>> for x, y in np.nditer([a,b], op_flags=[['readonly'],
- ... ['writeonly']]):
- ... y[...] = -x
- ...
- >>> b
- array([-1., -3., NA])
-
-When using the C-API version of the nditer, one must explicitly
-add the NPY_ITER_USE_MASKNA flag and take care to deal with the NA
-mask appropriately. In the Python exposure, this flag is added
-automatically.
-
-Planned Future Additions
-========================
-
-The NA support in 1.7 is fairly preliminary, and is focused on getting
-the basics solid. This particularly meant getting the API in C refined
-to a level where adding NA support to all of NumPy and to third party
-software using NumPy would be a reasonable task.
-
-The biggest missing feature within the core is supporting NA values with
-structured arrays. The design for this involves a mask slot for each
-field in the structured array, motivated by the fact that many important
-uses of structured arrays involve treating the structured fields like
-another dimension.
-
-Another feature that was discussed during the design process is the ability
-to support more than one NA value. The design created supports this multi-NA
-idea with the addition of a payload to the NA value and to the NA-mask.
-The API has been designed in such a way that adding this feature in a future
-release should be possible without changing existing API functions in any way.
-
-To see a more complete list of what is supported and unsupported in the
-1.7 release of NumPy, please refer to the release notes.
-
-During the design phase of this feature, two implementation approaches
-for NA values were discussed, called "mask" and "bitpattern". What
-has been implemented is the "mask" approach, but the design document,
-or "NEP", describes a way both approaches could co-operatively exist
-in NumPy, since each has both pros and cons. This design document is
-available in the file "doc/neps/missing-data.rst" of the NumPy source
-code.
View
1  doc/source/reference/arrays.rst
@@ -44,7 +44,6 @@ of also more complicated arrangements of data.
arrays.indexing
arrays.nditer
arrays.classes
- arrays.maskna
maskedarray
arrays.interface
arrays.datetime
View
114 doc/source/reference/c-api.array.rst
@@ -92,40 +92,6 @@ sub-types).
A synonym for PyArray_DESCR, named to be consistent with the
'dtype' usage within Python.
-.. cfunction:: npy_bool PyArray_HASMASKNA(PyArrayObject* arr)
-
- .. versionadded:: 1.7
-
- Returns true if the array has an NA-mask, false otherwise.
-
-.. cfunction:: PyArray_Descr *PyArray_MASKNA_DTYPE(PyArrayObject* arr)
-
- .. versionadded:: 1.7
-
- Returns a borrowed reference to the dtype property for the NA mask
- of the array, or NULL if the array has no NA mask. This function does
- not raise an exception when it returns NULL, it is simply returning
- the appropriate field.
-
-.. cfunction:: char *PyArray_MASKNA_DATA(PyArrayObject* arr)
-
- .. versionadded:: 1.7
-
- Returns a pointer to the raw data for the NA mask of the array,
- or NULL if the array has no NA mask. This function does
- not raise an exception when it returns NULL, it is simply returning
- the appropriate field.
-
-.. cfunction:: npy_intp *PyArray_MASKNA_STRIDES(PyArrayObject* arr)
-
- .. versionadded:: 1.7
-
- Returns a pointer to strides of the NA mask of the array, If the
- array has no NA mask, the values contained in the array will be
- invalid. The shape of the NA mask is identical to the shape of the
- array itself, so the number of strides is always the same as the
- number of array dimensions.
-
.. cfunction:: void PyArray_ENABLEFLAGS(PyArrayObject* arr, int flags)
.. versionadded:: 1.7
@@ -254,11 +220,6 @@ From scratch
provided *dims* and *strides* are copied into newly allocated
dimension and strides arrays for the new array object.
- Because the flags are ignored when *data* is NULL, you cannot
- create a new array from scratch with an NA mask. If one is desired,
- call the function :cfunc:`PyArray_AllocateMaskNA` after the array
- is created.
-
.. cfunction:: PyObject* PyArray_NewLikeArray(PyArrayObject* prototype, NPY_ORDER order, PyArray_Descr* descr, int subok)
.. versionadded:: 1.6
@@ -281,11 +242,6 @@ From scratch
*prototype* to create the new array, otherwise it will create a
base-class array.
- The newly allocated array does not have an NA mask even if the
- *prototype* provided does. If an NA mask is desired in the array,
- call the function :cfunc:`PyArray_AllocateMaskNA` after the array
- is created.
-
.. cfunction:: PyObject* PyArray_New(PyTypeObject* subtype, int nd, npy_intp* dims, int type_num, npy_intp* strides, void* data, int itemsize, int flags, PyObject* obj)
This is similar to :cfunc:`PyArray_DescrNew` (...) except you
@@ -475,31 +431,6 @@ From other objects
with, then an error is raised. If *op* is not already an array,
then this flag has no effect.
- .. cvar:: NPY_ARRAY_MASKNA
-
- .. versionadded:: 1.7
-
- Make sure the array has an NA mask associated with its data.
-
- .. cvar:: NPY_ARRAY_OWNMASKNA
-
- .. versionadded:: 1.7
-
- Make sure the array has an NA mask which it owns
- associated with its data.
-
- .. cvar:: NPY_ARRAY_ALLOWNA
-
- .. versionadded:: 1.7
-
- To prevent simple errors from slipping in, arrays with NA
- masks are not permitted to pass through by default. Instead
- an exception is raised indicating the operation doesn't support
- NA masks yet. In order to enable NA mask support, this flag
- must be passed in to allow the NA mask through, signalling that
- the later code is written appropriately to handle NA mask
- semantics.
-
.. cvar:: NPY_ARRAY_BEHAVED
:cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE`
@@ -1415,24 +1346,6 @@ or :cdata:`NPY_ARRAY_F_CONTIGUOUS` can be determined by the ``strides``,
would have returned an error because :cdata:`NPY_ARRAY_UPDATEIFCOPY`
would not have been possible.
-.. cvar:: NPY_ARRAY_MASKNA
-
- If this flag is enabled, the array has an NA mask associated with
- the data. C code which interacts with the NA mask must follow
- specific semantic rules about when to overwrite data and when not
- to. The mask can be accessed through the functions
- :cfunc:`PyArray_MASKNA_DTYPE`, :cfunc:`PyArray_MASKNA_DATA`, and
- :cfunc:`PyArray_MASKNA_STRIDES`.
-
-.. cvar:: NPY_ARRAY_OWNMASKNA
-
- If this flag is enabled, the array owns its own NA mask. If it is not
- enabled, the NA mask is a view into a different array's NA mask.
-
- In order to ensure that an array owns its own NA mask, you can
- call :cfunc:`PyArray_AllocateMaskNA` with the parameter *ownmaskna*
- set to 1.
-
:cfunc:`PyArray_UpdateFlags` (obj, flags) will update the ``obj->flags``
for ``flags`` which can be any of :cdata:`NPY_ARRAY_C_CONTIGUOUS`,
:cdata:`NPY_ARRAY_F_CONTIGUOUS`, :cdata:`NPY_ARRAY_ALIGNED`, or
@@ -2541,9 +2454,6 @@ Array Scalars
if so, returns the appropriate array scalar. It should be used
whenever 0-dimensional arrays could be returned to Python.
- If *arr* is a 0-dimensional NA-masked array with its value hidden,
- an instance of :ctype:`NpyNA *` is returned.
-
.. cfunction:: PyObject* PyArray_Scalar(void* data, PyArray_Descr* dtype, PyObject* itemsize)
Return an array scalar object of the given enumerated *typenum*
@@ -2756,19 +2666,6 @@ to.
. No matter what is returned, you must DECREF the object returned
by this routine in *address* when you are done with it.
- If the input is an array with NA support, this will either raise
- an error if it contains any NAs, or will make a copy of the array
- without NA support if it does not contain any NAs. Use the function
- :cfunc:`PyArray_AllowNAConverter` to support NA-arrays directly
- and more efficiently.
-
-.. cfunction:: int PyArray_AllowConverter(PyObject* obj, PyObject** address)
-
- This is the same as :cfunc:`PyArray_Converter`, but allows arrays
- with NA support to pass through untouched. This function was created
- so that the existing converter could raise errors appropriately
- for functions which have not been updated with NA support
-
.. cfunction:: int PyArray_OutputConverter(PyObject* obj, PyArrayObject** address)
This is a default converter for output arrays given to
@@ -2777,17 +2674,6 @@ to.
*obj*) is TRUE then it is returned in *\*address* without
incrementing its reference count.
- If the output is an array with NA support, this will raise an error.
- Use the function :cfunc:`PyArray_OutputAllowNAConverter` to support
- NA-arrays directly.
-
-.. cfunction:: int PyArray_OutputAllowNAConverter(PyObject* obj, PyArrayObject** address)
-
- This is the same as :cfunc:`PyArray_OutputConverter`, but allows arrays
- with NA support to pass through. This function was created
- so that the existing output converter could raise errors appropriately
- for functions which have not been updated with NA support
-
.. cfunction:: int PyArray_IntpConverter(PyObject* obj, PyArray_Dims* seq)
Convert any Python sequence, *obj*, smaller than :cdata:`NPY_MAXDIMS`
View
592 doc/source/reference/c-api.maskna.rst
@@ -1,592 +0,0 @@
-Array NA Mask API
-==================
-
-.. sectionauthor:: Mark Wiebe
-
-.. index::
- pair: maskna; C-API
- pair: C-API; maskna
-
-.. versionadded:: 1.7
-
-NA Masks in Arrays
-------------------
-
-NumPy supports the idea of NA (Not Available) missing values in its
-arrays. In the design document leading up to the implementation, two
-mechanisms for this were proposed, NA masks and NA bitpatterns. NA masks
-have been implemented as the first representation of these values. This
-mechanism supports working with NA values similar to what the R language
-provides, and when combined with views, allows one to temporarily mark
-elements as NA without affecting the original data.
-
-The C API has been updated with mechanisms to allow NumPy extensions
-to work with these masks, and this document provides some examples and
-reference for the NA mask-related functions.
-
-The NA Object
--------------
-
-The main *numpy* namespace in Python has a new object called *NA*.
-This is an instance of :ctype:`NpyNA`, which is a Python object
-representing an NA value. This object is analogous to the NumPy
-scalars, and is returned by :cfunc:`PyArray_Return` instead of
-a scalar where appropriate.
-
-The global *numpy.NA* object is accessible from C as :cdata:`Npy_NA`.
-This is an NA value with no data type or multi-NA payload. Use it
-just as you would Py_None, except use :cfunc:`NpyNA_Check` to
-see if an object is an :ctype:`NpyNA`, because :cdata:`Npy_NA` isn't
-the only instance of NA possible.
-
-If you want to see whether a general PyObject* is NA, you should
-use the API function :cfunc:`NpyNA_FromObject` with *suppress_error*
-set to true. If this returns NULL, the object is not an NA, and if
-it returns an NpyNA instance, the object is NA and you can then
-access its *dtype* and *payload* fields as needed.
-
-To make new :ctype:`NpyNA` objects, use
-:cfunc:`NpyNA_FromDTypeAndPayload`. The functions
-:cfunc:`NpyNA_GetDType`, :cfunc:`NpyNA_IsMultiNA`, and
-:cfunc:`NpyNA_GetPayload` provide access to the data members.
-
-Working With NA-Masked Arrays
------------------------------
-
-The starting point for many C-API functions which manipulate NumPy
-arrays is the function :cfunc:`PyArray_FromAny`. This function converts
-a general PyObject* object into a NumPy ndarray, based on options
-specified in the flags. To avoid surprises, this function does
-not allow NA-masked arrays to pass through by default.
-
-To allow third-party code to work with NA-masked arrays which contain
-no NAs, :cfunc:`PyArray_FromAny` will make a copy of the array into
-a new array without an NA-mask, and return that. This allows for
-proper interoperability in cases where it's possible until functions
-are updated to provide optimal code paths for NA-masked arrays.
-
-To update a function with NA-mask support, add the flag
-:cdata:`NPY_ARRAY_ALLOWNA` when calling :cfunc:`PyArray_FromAny`.
-This allows NA-masked arrays to pass through untouched, and will
-convert PyObject lists containing NA values into NA-masked arrays
-instead of the alternative of switching to object arrays.
-
-To check whether an array has an NA-mask, use the function
-:cfunc:`PyArray_HASMASKNA`, which checks the appropriate flag.
-There are a number of things that one will typically want to do
-when encountering an NA-masked array. We'll go through a few
-of these cases.
-
-Forbidding Any NA Values
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-The simplest case is to forbid any NA values. Note that it is better
-to still be aware of the NA mask and explicitly test for NA values
-than to leave out the :cdata:`NPY_ARRAY_ALLOWNA`, because it is possible
-to avoid the extra copy that :cfunc:`PyArray_FromAny` will make. The
-check for NAs will go something like this::
-
- PyArrayObject *arr = ...;
- int containsna;
-
- /* ContainsNA checks HASMASKNA() for you */
- containsna = PyArray_ContainsNA(arr, NULL, NULL);
- /* Error case */
- if (containsna < 0) {
- return NULL;
- }
- /* If it found an NA */
- else if (containsna) {
- PyErr_SetString(PyExc_ValueError,
- "this operation does not support arrays with NA values");
- return NULL;
- }
-
-After this check, you can be certain that the array doesn't contain any
-NA values, and can proceed accordingly. For example, if you iterate
-over the elements of the array, you may pass the flag
-:cdata:`NPY_ITER_IGNORE_MASKNA` to iterate over the data without
-touching the NA-mask at all.
-
-Manipulating NA Values
-~~~~~~~~~~~~~~~~~~~~~~
-
-The semantics of the NA-mask demand that whenever an array element
-is hidden by the NA-mask, no computations are permitted to modify
-the data backing that element. The :ctype:`NpyIter` provides
-a number of flags to assist with visiting both the array data
-and the mask data simultaneously, and preserving the masking semantics
-even when buffering is required.
-
-The main flag for iterating over NA-masked arrays is
-:cdata:`NPY_ITER_USE_MASKNA`. For each iterator operand which has this
-flag specified, a new operand is added to the end of the iterator operand
-list, and is set to iterate over the original operand's NA-mask. Operands
-which do not have an NA mask are permitted as well when they are flagged
-as read-only. The new operand in this case points to a single exposed
-mask value and all its strides are zero. The latter feature is useful
-when combining multiple read-only inputs, where some of them have masks.
-
-Accumulating NA Values
-~~~~~~~~~~~~~~~~~~~~~~
-
-More complex operations, like the NumPy ufunc reduce functions, need
-to take extra care to follow the masking semantics. If we accumulate
-the NA mask and the data values together, we could discover half way
-through that the output is NA, and that we have violated the contract
-to never change the underlying output value when it is being assigned
-NA.
-
-The solution to this problem is to first accumulate the NA-mask as necessary
-to produce the output's NA-mask, then accumulate the data values without
-touching NA-masked values in the output. The parameter *preservena* in
-functions like :cfunc:`PyArray_AssignArray` can assist when initializing
-values in such an algorithm.
-
-Example NA-Masked Operation in C
---------------------------------
-
-As an example, let's implement a simple binary NA-masked operation
-for the double dtype. We'll make a divide operation which turns
-divide by zero into NA instead of Inf or NaN.
-
-To start, we define the function prototype and some basic
-:ctype:`NpyIter` boilerplate setup. We'll make a function which
-supports an optional *out* parameter, which may be NULL.::
-
- static PyArrayObject*
- SpecialDivide(PyArrayObject* a, PyArrayObject* b, PyArrayObject *out)
- {
- NpyIter *iter = NULL;
- PyArrayObject *op[3];
- PyArray_Descr *dtypes[3];
- npy_uint32 flags, op_flags[3];
-
- /* Iterator construction parameters */
- op[0] = a;
- op[1] = b;
- op[2] = out;
-
- dtypes[0] = PyArray_DescrFromType(NPY_DOUBLE);
- if (dtypes[0] == NULL) {
- return NULL;
- }
- dtypes[1] = dtypes[0];
- dtypes[2] = dtypes[0];
-
- flags = NPY_ITER_BUFFERED |
- NPY_ITER_EXTERNAL_LOOP |
- NPY_ITER_GROWINNER |
- NPY_ITER_REFS_OK |
- NPY_ITER_ZEROSIZE_OK;
-
- /* Every operand gets the flag NPY_ITER_USE_MASKNA */
- op_flags[0] = NPY_ITER_READONLY |
- NPY_ITER_ALIGNED |
- NPY_ITER_USE_MASKNA;
- op_flags[1] = op_flags[0];
- op_flags[2] = NPY_ITER_WRITEONLY |
- NPY_ITER_ALIGNED |
- NPY_ITER_USE_MASKNA |
- NPY_ITER_NO_BROADCAST |
- NPY_ITER_ALLOCATE;
-
- iter = NpyIter_MultiNew(3, op, flags, NPY_KEEPORDER,
- NPY_SAME_KIND_CASTING, op_flags, dtypes);
- /* Don't need the dtype reference anymore */
- Py_DECREF(dtypes[0]);
- if (iter == NULL) {
- return NULL;
- }
-
-At this point, the input operands have been validated according to
-the casting rule, the shapes of the arrays have been broadcast together,
-and any buffering necessary has been prepared. This means we can
-dive into the inner loop of this function.::
-
- ...
- if (NpyIter_GetIterSize(iter) > 0) {
- NpyIter_IterNextFunc *iternext;
- char **dataptr;
- npy_intp *stridesptr, *countptr;
-
- /* Variables needed for looping */
- iternext = NpyIter_GetIterNext(iter, NULL);
- if (iternext == NULL) {
- NpyIter_Deallocate(iter);
- return NULL;
- }
- dataptr = NpyIter_GetDataPtrArray(iter);
- stridesptr = NpyIter_GetInnerStrideArray(iter);
- countptr = NpyIter_GetInnerLoopSizePtr(iter);
-
-The loop gets a bit messy when dealing with NA-masks, because it
-doubles the number of operands being processed in the iterator. Here
-we are naming things clearly so that the content of the innermost loop
-can be easy to work with.::
-
- ...
- do {
- /* Data pointers and strides needed for innermost loop */
- char *data_a = dataptr[0], *data_b = dataptr[1];
- char *data_out = dataptr[2];
- char *maskna_a = dataptr[3], *maskna_b = dataptr[4];
- char *maskna_out = dataptr[5];
- npy_intp stride_a = stridesptr[0], stride_b = stridesptr[1];
- npy_intp stride_out = strides[2];
- npy_intp maskna_stride_a = stridesptr[3];
- npy_intp maskna_stride_b = stridesptr[4];
- npy_intp maskna_stride_out = stridesptr[5];
- npy_intp i, count = *countptr;
-
- for (i = 0; i < count; ++i) {
-
-Here is the code for performing one special division. We use
-the functions :cfunc:`NpyMaskValue_IsExposed` and
-:cfunc:`NpyMaskValue_Create` to work with the masks, in order to be
-as general as possible. These are inline functions, and the compiler
-optimizer should be able to produce the same result as if you performed
-these operations directly inline here.::
-
- ...
- /* If neither of the inputs are NA */
- if (NpyMaskValue_IsExposed((npy_mask)*maskna_a) &&
- NpyMaskValue_IsExposed((npy_mask)*maskna_b)) {
- double a_val = *(double *)data_a;
- double b_val = *(double *)data_b;
- /* Do the divide if 'b' isn't zero */
- if (b_val != 0.0) {
- *(double *)data_out = a_val / b_val;
- /* Need to also set this element to exposed */
- *maskna_out = NpyMaskValue_Create(1, 0);
- }
- /* Otherwise output an NA without touching its data */
- else {
- *maskna_out = NpyMaskValue_Create(0, 0);
- }
- }
- /* Turn the output into NA without touching its data */
- else {
- *maskna_out = NpyMaskValue_Create(0, 0);
- }
-
- data_a += stride_a;
- data_b += stride_b;
- data_out += stride_out;
- maskna_a += maskna_stride_a;
- maskna_b += maskna_stride_b;
- maskna_out += maskna_stride_out;
- }
- } while (iternext(iter));
- }
-
-A little bit more boilerplate for returning the result from the iterator,
-and the function is done.::
-
- ...
- if (out == NULL) {
- out = NpyIter_GetOperandArray(iter)[2];
- }
- Py_INCREF(out);
- NpyIter_Deallocate(iter);
-
- return out;
- }
-
-To run this example, you can create a simple module with a C-file spdiv_mod.c
-consisting of::
-
- #include <Python.h>
- #include <numpy/arrayobject.h>
-
- /* INSERT SpecialDivide source code here */
-
- static PyObject *
- spdiv(PyObject *self, PyObject *args, PyObject *kwds)
- {
- PyArrayObject *a, *b, *out = NULL;
- static char *kwlist[] = {"a", "b", "out", NULL};
-
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "O&O&|O&", kwlist,
- &PyArray_AllowNAConverter, &a,
- &PyArray_AllowNAConverter, &b,
- &PyArray_OutputAllowNAConverter, &out)) {
- return NULL;
- }
-
- /*
- * The usual NumPy way is to only use PyArray_Return when
- * the 'out' parameter is not provided.
- */
- if (out == NULL) {
- return PyArray_Return(SpecialDivide(a, b, out));
- }
- else {
- return (PyObject *)SpecialDivide(a, b, out);
- }
- }
-
- static PyMethodDef SpDivMethods[] = {
- {"spdiv", (PyCFunction)spdiv, METH_VARARGS | METH_KEYWORDS, NULL},
- {NULL, NULL, 0, NULL}
- };
-
-
- PyMODINIT_FUNC initspdiv_mod(void)
- {
- PyObject *m;
-
- m = Py_InitModule("spdiv_mod", SpDivMethods);
- if (m == NULL) {
- return;
- }
-
- /* Make sure NumPy is initialized */
- import_array();
- }
-
-Create a setup.py file like::
-
- #!/usr/bin/env python
- def configuration(parent_package='',top_path=None):
- from numpy.distutils.misc_util import Configuration
- config = Configuration('.',parent_package,top_path)
- config.add_extension('spdiv_mod',['spdiv_mod.c'])
- return config
-
- if __name__ == "__main__":
- from numpy.distutils.core import setup
- setup(configuration=configuration)
-
-With these two files in a directory by itself, run::
-
- $ python setup.py build_ext --inplace
-
-and the file spdiv_mod.so (or .dll) will be placed in the same directory.
-Now you can try out this sample, to see how it behaves.::
-
- >>> import numpy as np
- >>> from spdiv_mod import spdiv
-
-Because we used :cfunc:`PyArray_Return` when wrapping SpecialDivide,
-it returns scalars like any typical NumPy function does::
-
- >>> spdiv(1, 2)
- 0.5
- >>> spdiv(2, 0)
- NA(dtype='float64')
- >>> spdiv(np.NA, 1.5)
- NA(dtype='float64')
-
-Here we can see how NAs propagate, and how 0 in the output turns into NA
-as desired.::
-
- >>> a = np.arange(6)
- >>> b = np.array([0,np.NA,0,2,1,0])
- >>> spdiv(a, b)
- array([ NA, NA, NA, 1.5, 4. , NA])
-
-Finally, we can see the masking behavior by creating a masked
-view of an array. The ones in *c_orig* are preserved whereever
-NA got assigned.::
-
- >>> c_orig = np.ones(6)
- >>> c = c_orig.view(maskna=True)
- >>> spdiv(a, b, out=c)
- array([ NA, NA, NA, 1.5, 4. , NA])
- >>> c_orig
- array([ 1. , 1. , 1. , 1.5, 4. , 1. ])
-
-NA Object Data Type
--------------------
-
-.. ctype:: NpyNA
-
- This is the C object corresponding to objects of type
- numpy.NAType. The fields themselves are hidden from consumers of the
- API, you must use the functions provided to create new NA objects
- and get their properties.
-
- This object contains two fields, a :ctype:`PyArray_Descr *` dtype
- which is either NULL or indicates the data type the NA represents,
- and a payload which is there for the future addition of multi-NA support.
-
-.. cvar:: Npy_NA
-
- This is a global singleton, similar to Py_None, which is the
- *numpy.NA* object. Note that unlike Py_None, multiple NAs may be
- created, for instance with different multi-NA payloads or with
- different dtypes. If you want to return an NA with no payload
- or dtype, return a new reference to Npy_NA.
-
-NA Object Functions
--------------------
-
-.. cfunction:: NpyNA_Check(obj)
-
- Evaluates to true if *obj* is an instance of :ctype:`NpyNA`.
-
-.. cfunction:: PyArray_Descr* NpyNA_GetDType(NpyNA* na)
-
- Returns the *dtype* field of the NA object, which is NULL when
- the NA has no dtype. Does not raise an error.
-
-.. cfunction:: npy_bool NpyNA_IsMultiNA(NpyNA* na)
-
- Returns true if the NA has a multi-NA payload, false otherwise.
-
-.. cfunction:: int NpyNA_GetPayload(NpyNA* na)
-
- Gets the multi-NA payload of the NA, or 0 if *na* doesn't have
- a multi-NA payload.
-
-.. cfunction:: NpyNA* NpyNA_FromObject(PyObject* obj, int suppress_error)
-
- If *obj* represents an object which is NA, for example if it
- is an :ctype:`NpyNA`, or a zero-dimensional NA-masked array with
- its value hidden by the mask, returns a new reference to an
- :ctype:`NpyNA` object representing *obj*. Otherwise returns
- NULL.
-
- If *suppress_error* is true, this function doesn't raise an exception
- when the input isn't NA and it returns NULL, otherwise it does.
-
-.. cfunction:: NpyNA* NpyNA_FromDTypeAndPayload(PyArray_Descr *dtype, int multina, int payload)
-
-
- Constructs a new :ctype:`NpyNA` instance with the specified *dtype*
- and *payload*. For an NA with no dtype, provide NULL in *dtype*.
-
- Until multi-NA is implemented, just pass 0 for both *multina*
- and *payload*.
-
-NA Mask Functions
------------------
-
-A mask dtype can be one of three different possibilities. It can
-be :cdata:`NPY_BOOL`, :cdata:`NPY_MASK`, or a struct dtype whose
-fields are all mask dtypes.
-
-A mask of :cdata:`NPY_BOOL` can just indicate True, with underlying
-value 1, for an element that is exposed, and False, with underlying
-value 0, for an element that is hidden.
-
-A mask of :cdata:`NPY_MASK` can additionally carry a payload which
-is a value from 0 to 127. This allows for missing data implementations
-based on such masks to support multiple reasons for data being missing.
-
-A mask of a struct dtype can only pair up with another struct dtype
-with the same field names. In this way, each field of the mask controls
-the masking for the corresponding field in the associated data array.
-
-Inline functions to work with masks are as follows.
-
-.. cfunction:: npy_bool NpyMaskValue_IsExposed(npy_mask mask)
-
- Returns true if the data element corresponding to the mask element
- can be modified, false if not.
-
-.. cfunction:: npy_uint8 NpyMaskValue_GetPayload(npy_mask mask)
-
- Returns the payload contained in the mask. The return value
- is between 0 and 127.
-
-.. cfunction:: npy_mask NpyMaskValue_Create(npy_bool exposed, npy_int8 payload)
-
- Creates a mask from a flag indicating whether the element is exposed
- or not and a payload value.
-
-NA Mask Array Functions
------------------------
-
-.. cfunction:: int PyArray_AllocateMaskNA(PyArrayObject *arr, npy_bool ownmaskna, npy_bool multina, npy_mask defaultmask)
-
- Allocates an NA mask for the array *arr* if necessary. If *ownmaskna*
- if false, it only allocates an NA mask if none exists, but if
- *ownmaskna* is true, it also allocates one if the NA mask is a view
- into another array's NA mask. Here are the two most common usage
- patterns::
-
- /* Use this to make sure 'arr' has an NA mask */
- if (PyArray_AllocateMaskNA(arr, 0, 0, 1) < 0) {
- return NULL;
- }
-
- /* Use this to make sure 'arr' owns an NA mask */
- if (PyArray_AllocateMaskNA(arr, 1, 0, 1) < 0) {
- return NULL;
- }
-
- The parameter *multina* is provided for future expansion, when
- mult-NA support is added to NumPy. This will affect the dtype of
- the NA mask, which currently must be always NPY_BOOL, but will be
- NPY_MASK for arrays multi-NA when this is implemented.
-
- When a new NA mask is allocated, and the mask needs to be filled,
- it uses the value *defaultmask*. In nearly all cases, this should be set
- to 1, indicating that the elements are exposed. If a mask is allocated
- just because of *ownmaskna*, the existing mask values are copied
- into the newly allocated mask.
-
- This function returns 0 for success, -1 for failure.
-
-.. cfunction:: npy_bool PyArray_HasNASupport(PyArrayObject *arr)
-
- Returns true if *arr* is an array which supports NA. This function
- exists because the design for adding NA proposed two mechanisms
- for NAs in NumPy, NA masks and NA bitpatterns. Currently, just
- NA masks have been implemented, but when NA bitpatterns are implemented
- this would return true for arrays with an NA bitpattern dtype as well.
-
-.. cfunction:: int PyArray_ContainsNA(PyArrayObject *arr, PyArrayObject *wheremask, npy_bool *whichna)
-
- Checks whether the array *arr* contains any NA values.
-
- If *wheremask* is non-NULL, it must be an NPY_BOOL mask which can
- broadcast onto *arr*. Whereever the where mask is True, *arr*
- is checked for NA, and whereever it is False, the *arr* value is
- ignored.
-
- The parameter *whichna* is provided for future expansion to multi-NA
- support. When implemented, this parameter will be a 128 element
- array of npy_bool, with the value True for the NA values that are
- being looked for.
-
- This function returns 1 when the array contains NA values, 0 when
- it does not, and -1 when a error has occurred.
-
-.. cfunction:: int PyArray_AssignNA(PyArrayObject *arr, NpyNA *na, PyArrayObject *wheremask, npy_bool preservena, npy_bool *preservewhichna)
-
- Assigns the given *na* value to elements of *arr*.
-
- If *wheremask* is non-NULL, it must be an NPY_BOOL array broadcastable
- onto *arr*, and only elements of *arr* with a corresponding value
- of True in *wheremask* will have *na* assigned.
-
- The parameters *preservena* and *preservewhichna* are provided for
- future expansion to multi-NA support. With a single NA value, one
- NA cannot be distinguished from another, so preserving NA values
- does not make sense. With multiple NA values, preserving NA values
- becomes an important concept because that implies not overwriting the
- multi-NA payloads. The parameter *preservewhichna* will be a 128 element
- array of npy_bool, indicating which NA payloads to preserve.
-
- This function returns 0 for success, -1 for failure.
-
-.. cfunction:: int PyArray_AssignMaskNA(PyArrayObject *arr, npy_mask maskvalue, PyArrayObject *wheremask, npy_bool preservena, npy_bool *preservewhichna)
-
- Assigns the given NA mask *maskvalue* to elements of *arr*.
-
- If *wheremask* is non-NULL, it must be an NPY_BOOL array broadcastable
- onto *arr*, and only elements of *arr* with a corresponding value
- of True in *wheremask* will have the NA *maskvalue* assigned.
-
- The parameters *preservena* and *preservewhichna* are provided for
- future expansion to multi-NA support. With a single NA value, one
- NA cannot be distinguished from another, so preserving NA values
- does not make sense. With multiple NA values, preserving NA values
- becomes an important concept because that implies not overwriting the
- multi-NA payloads. The parameter *preservewhichna* will be a 128 element
- array of npy_bool, indicating which NA payloads to preserve.
-
- This function returns 0 for success, -1 for failure.
View
1  doc/source/reference/c-api.rst
@@ -45,7 +45,6 @@ code.
c-api.dtype
c-api.array
c-api.iterator
- c-api.maskna
c-api.ufunc
c-api.generalized-ufuncs
c-api.coremath
View
28 doc/source/reference/routines.polynomials.classes.rst
@@ -322,31 +322,3 @@ illustrated below for a fit to a noisy sin curve.
>>> p.window
array([-1., 1.])
>>> plt.show()
-
-The fit will ignore data points masked with NA. We demonstrate this with
-the previous example, but add an outlier that messes up the fit, then mask
-it out.
-
-.. plot::
-
- >>> import numpy as np
- >>> import matplotlib.pyplot as plt
- >>> from numpy.polynomial import Chebyshev as T
- >>> np.random.seed(11)
- >>> x = np.linspace(0, 2*np.pi, 20)
- >>> y = np.sin(x) + np.random.normal(scale=.1, size=x.shape)
- >>> y[10] = 2
- >>> p = T.fit(x, y, 5)
- >>> plt.plot(x, y, 'o')
- [<matplotlib.lines.Line2D object at 0x2136c10>]
- >>> xx, yy = p.linspace()
- >>> plt.plot(xx, yy, lw=2, label="unmasked")
- [<matplotlib.lines.Line2D object at 0x1cf2890>]
- >>> ym = y.view(maskna=1)
- >>> ym[10] = np.NA
- >>> p = T.fit(x, ym, 5)
- >>> xx, yy = p.linspace()
- >>> plt.plot(xx, yy, lw=2, label="masked")
- >>> plt.legend(loc="upper right")
- <matplotlib.legend.Legend object at 0x3b3ee10>
- >>> plt.show()
View
1  doc/source/reference/routines.rst
@@ -34,7 +34,6 @@ indentation.
routines.linalg
routines.logic
routines.ma
- routines.maskna
routines.math
routines.matlib
routines.numarray
View
133 numpy/add_newdocs.py
@@ -204,8 +204,6 @@
copies those elements indicated by this mask.
* 'writemasked' indicates that only elements where the chosen
'arraymask' operand is True will be written to.
- * 'use_maskna' indicates that this operand should be treated
- like an NA-masked array.
op_dtypes : dtype or tuple of dtype(s), optional
The required data type(s) of the operands. If copying or buffering
is enabled, the data will be converted to/from their original types.
@@ -640,7 +638,7 @@ def luf(lamdaexpr, *args, **kwargs):
add_newdoc('numpy.core.multiarray', 'array',
"""
- array(object, dtype=None, copy=True, order=None, subok=False, ndmin=0, maskna=None, ownmaskna=False)
+ array(object, dtype=None, copy=True, order=None, subok=False, ndmin=0)
Create an array.
@@ -676,25 +674,6 @@ def luf(lamdaexpr, *args, **kwargs):
Specifies the minimum number of dimensions that the resulting
array should have. Ones will be pre-pended to the shape as
needed to meet this requirement.
- maskna : bool or None, optional
- If this is set to True, it forces the array to have an NA mask.
- If the input is an array without a mask, this means a view with
- an NA mask is created. If the input is an array with a mask, the
- mask is preserved as-is.
-
- If this is set to False, it forces the array to not have an NA
- mask. If the input is an array with a mask, and has no NA values,
- it will create a copy of the input without an NA mask.
-
- .. versionadded:: 1.7.0
-
- ownmaskna : bool, optional
- If this is set to True, forces the array to have a mask which
- it owns. It may still return a view of the data from the input,
- but the result will always own its own mask.
-
- .. versionadded:: 1.7.0
-
Returns
-------
@@ -705,11 +684,6 @@ def luf(lamdaexpr, *args, **kwargs):
--------
empty, empty_like, zeros, zeros_like, ones, ones_like, fill
- Notes
- -----
- The `maskna` and `ownmaskna` keywords are *experimental* in the 1.7
- release; their behavior may change in future versions.
-
Examples
--------
>>> np.array([1, 2, 3])
@@ -769,8 +743,6 @@ def luf(lamdaexpr, *args, **kwargs):
order : {'C', 'F'}, optional
Whether to store multi-dimensional data in C (row-major) or
Fortran (column-major) order in memory.
- maskna : boolean
- If this is true, the returned array will have an NA mask.
See Also
--------
@@ -919,35 +891,6 @@ def luf(lamdaexpr, *args, **kwargs):
""")
-add_newdoc('numpy.core.multiarray', 'isna',
- """
- isna(a)
-
- Returns an array with True for each element of *a* that is NA.
-
- Parameters
- ----------
- a : array_like
- The array for which to check for NA.
-
- Returns
- -------
- result : bool or array of bool
- Number of non-zero values in the array.
-
- Examples
- --------
- >>> np.isna(np.NA)
- True
- >>> np.isna(1.5)
- False
- >>> np.isna(np.nan)
- False
- >>> a = np.array([0, np.NA, 3.5, np.NA])
- >>> np.isna(a)
- array([False, True, False, True], dtype=bool)
- """)
-
add_newdoc('numpy.core.multiarray', 'count_nonzero',
"""
count_nonzero(a)
@@ -962,9 +905,6 @@ def luf(lamdaexpr, *args, **kwargs):
Axis or axes along which a reduction is performed.
The default (`axis` = None) is perform a reduction over all
the dimensions of the input array.
- skipna : bool, optional
- If this is set to True, any NA elements in the array are skipped
- instead of propagating.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -992,60 +932,6 @@ def luf(lamdaexpr, *args, **kwargs):
[3]])
""")
-add_newdoc('numpy.core.multiarray', 'count_reduce_items',
- """
- count_reduce_items(arr, axis=None, skipna=False, keepdims=False)
-
- Counts the number of items a reduction with the same `axis`
- and `skipna` parameter values would use. The purpose of this
- function is for the creation of reduction operations
- which use the item count, such as :func:`mean`.
-
- When `skipna` is False or `arr` doesn't have an NA mask,
- the result is simply the product of the reduction axis
- sizes, returned as a single scalar.
-
- Parameters
- ----------
- arr : array_like
- The array for which to count the reduce items.
- axis : None or int or tuple of ints, optional
- Axis or axes along which a reduction is performed.
- The default (`axis` = None) is perform a reduction over all
- the dimensions of the input array.
- skipna : bool, optional
- If this is set to True, any NA elements in the array are not
- counted. The only time this function does any actual counting
- instead of a cheap multiply of a few sizes is when `skipna` is
- true and `arr` has an NA mask.
- keepdims : bool, optional
- If this is set to True, the axes which are reduced are left
- in the result as dimensions with size one. With this option,
- the result will broadcast correctly against the original `arr`.
-
- Returns
- -------
- count : intp or array of intp
- Number of items that would be used in a reduction with the
- same `axis` and `skipna` parameter values.
-
- Examples
- --------
- >>> a = np.array([[1,np.NA,1], [1,1,np.NA]])
-
- >>> np.count_reduce_items(a)
- 6
- >>> np.count_reduce_items(a, skipna=True)
- 4
- >>> np.sum(a, skipna=True)
- 4
-
- >>> np.count_reduce_items(a, axis=0, skipna=True)
- array([2, 1, 1])
- >>> np.sum(a, axis=0, skipna=True)
- array([2, 1, 1])
- """)
-
add_newdoc('numpy.core.multiarray','set_typeDict',
"""set_typeDict(dict)
@@ -1409,7 +1295,7 @@ def luf(lamdaexpr, *args, **kwargs):
add_newdoc('numpy.core.multiarray', 'arange',
"""
- arange([start,] stop[, step,], dtype=None, maskna=False)
+ arange([start,] stop[, step,], dtype=None)
Return evenly spaced values within a given interval.
@@ -1438,8 +1324,6 @@ def luf(lamdaexpr, *args, **kwargs):
dtype : dtype
The type of the output array. If `dtype` is not given, infer the data
type from the other input arguments.
- maskna : boolean
- If this is true, the returned array will have an NA mask.
Returns
-------
@@ -3320,7 +3204,7 @@ def luf(lamdaexpr, *args, **kwargs):
add_newdoc('numpy.core.multiarray', 'ndarray', ('copy',
"""
- a.copy(order='C', maskna=None)
+ a.copy(order='C')
Return a copy of the array.
@@ -3331,10 +3215,6 @@ def luf(lamdaexpr, *args, **kwargs):
'F' means F-order, 'A' means 'F' if `a` is Fortran contiguous,
'C' otherwise. 'K' means match the layout of `a` as closely
as possible.
- maskna : bool, optional
- If specifies, forces the copy to have or to not have an
- NA mask. This is a way to remove an NA mask from an array
- while making a copy.
See also
--------
@@ -5470,7 +5350,7 @@ def luf(lamdaexpr, *args, **kwargs):
add_newdoc('numpy.core', 'ufunc', ('reduce',
"""
- reduce(a, axis=0, dtype=None, out=None, skipna=False, keepdims=False)
+ reduce(a, axis=0, dtype=None, out=None, keepdims=False)
Reduces `a`'s dimension by one, by applying ufunc along one axis.
@@ -5515,11 +5395,6 @@ def luf(lamdaexpr, *args, **kwargs):
out : ndarray, optional
A location into which the result is stored. If not provided, a
freshly-allocated array is returned.
- skipna : bool, optional
- If this is set to True, the reduction is done as if any NA elements
- were not counted in the array. The default, False, causes the
- NA values to propagate, so if any element in a set of elements
- being reduced is NA, the result will be NA.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
View
64 numpy/core/_methods.py
@@ -5,34 +5,51 @@
from numpy.core import umath as um
from numpy.core.numeric import asanyarray
-def _amax(a, axis=None, out=None, skipna=False, keepdims=False):
+def _amax(a, axis=None, out=None, keepdims=False):
return um.maximum.reduce(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def _amin(a, axis=None, out=None, skipna=False, keepdims=False):
+def _amin(a, axis=None, out=None, keepdims=False):
return um.minimum.reduce(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def _sum(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def _sum(a, axis=None, dtype=None, out=None, keepdims=False):
return um.add.reduce(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def _prod(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def _prod(a, axis=None, dtype=None, out=None, keepdims=False):
return um.multiply.reduce(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
-
-def _mean(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+ out=out, keepdims=keepdims)
+
+def _any(a, axis=None, dtype=None, out=None, keepdims=False):
+ return um.logical_or.reduce(a, axis=axis, dtype=dtype, out=out,
+ keepdims=keepdims)
+
+def _all(a, axis=None, dtype=None, out=None, keepdims=False):
+ return um.logical_and.reduce(a, axis=axis, dtype=dtype, out=out,
+ keepdims=keepdims)
+
+def _count_reduce_items(arr, axis):
+ if axis is None:
+ axis = tuple(xrange(arr.ndim))
+ if not isinstance(axis, tuple):
+ axis = (axis,)
+ items = 1
+ for ax in axis:
+ items *= arr.shape[ax]
+ return items
+
+def _mean(a, axis=None, dtype=None, out=None, keepdims=False):
arr = asanyarray(a)
# Upgrade bool, unsigned int, and int to float64
if dtype is None and arr.dtype.kind in ['b','u','i']:
ret = um.add.reduce(arr, axis=axis, dtype='f8',
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
else:
ret = um.add.reduce(arr, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
- rcount = mu.count_reduce_items(arr, axis=axis,
- skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
+ rcount = _count_reduce_items(arr, axis)
if isinstance(ret, mu.ndarray):
ret = um.true_divide(ret, rcount,
out=ret, casting='unsafe', subok=False)
@@ -41,18 +58,15 @@ def _mean(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
return ret
def _var(a, axis=None, dtype=None, out=None, ddof=0,
- skipna=False, keepdims=False):
+ keepdims=False):
arr = asanyarray(a)
# First compute the mean, saving 'rcount' for reuse later
if dtype is None and arr.dtype.kind in ['b','u','i']:
- arrmean = um.add.reduce(arr, axis=axis, dtype='f8',
- skipna=skipna, keepdims=True)
+ arrmean = um.add.reduce(arr, axis=axis, dtype='f8', keepdims=True)
else:
- arrmean = um.add.reduce(arr, axis=axis, dtype=dtype,
- skipna=skipna, keepdims=True)
- rcount = mu.count_reduce_items(arr, axis=axis,
- skipna=skipna, keepdims=True)
+ arrmean = um.add.reduce(arr, axis=axis, dtype=dtype, keepdims=True)
+ rcount = _count_reduce_items(arr, axis)
if isinstance(arrmean, mu.ndarray):
arrmean = um.true_divide(arrmean, rcount,
out=arrmean, casting='unsafe', subok=False)
@@ -69,8 +83,7 @@ def _var(a, axis=None, dtype=None, out=None, ddof=0,
x = um.multiply(x, x, out=x)
# add.reduce((arr - arrmean) ** 2, axis)
- ret = um.add.reduce(x, axis=axis, dtype=dtype, out=out,
- skipna=skipna, keepdims=keepdims)
+ ret = um.add.reduce(x, axis=axis, dtype=dtype, out=out, keepdims=keepdims)
# add.reduce((arr - arrmean) ** 2, axis) / (n - ddof)
if not keepdims and isinstance(rcount, mu.ndarray):
@@ -84,10 +97,9 @@ def _var(a, axis=None, dtype=None, out=None, ddof=0,
return ret
-def _std(a, axis=None, dtype=None, out=None, ddof=0,
- skipna=False, keepdims=False):
+def _std(a, axis=None, dtype=None, out=None, ddof=0, keepdims=False):
ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
- skipna=skipna, keepdims=keepdims)
+ keepdims=keepdims)
if isinstance(ret, mu.ndarray):
ret = um.sqrt(ret, out=ret)
View
115 numpy/core/arrayprint.py
@@ -15,7 +15,7 @@
import sys
import numerictypes as _nt
from umath import maximum, minimum, absolute, not_equal, isnan, isinf
-from multiarray import format_longfloat, datetime_as_string, datetime_data, isna
+from multiarray import format_longfloat, datetime_as_string, datetime_data
from fromnumeric import ravel
@@ -29,7 +29,6 @@ def product(x, y): return x*y
_line_width = 75
_nan_str = 'nan'
_inf_str = 'inf'
-_na_str = 'NA'
_formatter = None # formatting function for array elements
if sys.version_info[0] >= 3:
@@ -37,7 +36,7 @@ def product(x, y): return x*y
def set_printoptions(precision=None, threshold=None, edgeitems=None,
linewidth=None, suppress=None,
- nanstr=None, infstr=None, nastr=None,
+ nanstr=None, infstr=None,
formatter=None):
"""
Set printing options.
@@ -65,8 +64,6 @@ def set_printoptions(precision=None, threshold=None, edgeitems=None,
String representation of floating point not-a-number (default nan).
infstr : str, optional
String representation of floating point infinity (default inf).
- nastr : str, optional
- String representation of NA missing value (default NA).
formatter : dict of callables, optional
If not None, the keys should indicate the type(s) that the respective
formatting function applies to. Callables should return a string.
@@ -144,7 +141,7 @@ def set_printoptions(precision=None, threshold=None, edgeitems=None,
global _summaryThreshold, _summaryEdgeItems, _float_output_precision, \
_line_width, _float_output_suppress_small, _nan_str, _inf_str, \
- _na_str, _formatter
+ _formatter
if linewidth is not None:
_line_width = linewidth
if threshold is not None:
@@ -159,8 +156,6 @@ def set_printoptions(precision=None, threshold=None, edgeitems=None,
_nan_str = nanstr
if infstr is not None:
_inf_str = infstr
- if nastr is not None:
- _na_str = nastr
_formatter = formatter
def get_printoptions():
@@ -195,7 +190,6 @@ def get_printoptions():
suppress=_float_output_suppress_small,
nanstr=_nan_str,
infstr=_inf_str,
- nastr=_na_str,
formatter=_formatter)
return d
@@ -219,19 +213,14 @@ def _leading_trailing(a):
return b
def _boolFormatter(x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- elif x:
+ if x:
return ' True'
else:
return 'False'
def repr_format(x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- else:
- return repr(x)
+ return repr(x)
def _array2string(a, max_line_width, precision, suppress_small, separator=' ',
prefix="", formatter=None):
@@ -437,20 +426,17 @@ def array2string(a, max_line_width=None, precision=None,
if a.shape == ():
x = a.item()
- if isna(x):
- lst = str(x).replace('NA', _na_str, 1)
- else:
- try:
- lst = a._format(x)
- msg = "The `_format` attribute is deprecated in Numpy " \
- "2.0 and will be removed in 2.1. Use the " \
- "`formatter` kw instead."
- import warnings
- warnings.warn(msg, DeprecationWarning)
- except AttributeError:
- if isinstance(x, tuple):
- x = _convert_arrays(x)
- lst = style(x)
+ try:
+ lst = a._format(x)
+ msg = "The `_format` attribute is deprecated in Numpy " \
+ "2.0 and will be removed in 2.1. Use the " \
+ "`formatter` kw instead."
+ import warnings
+ warnings.warn(msg, DeprecationWarning)
+ except AttributeError:
+ if isinstance(x, tuple):
+ x = _convert_arrays(x)
+ lst = style(x)
elif reduce(product, a.shape) == 0:
# treat as a null array if any of shape elements == 0
lst = "[]"
@@ -553,17 +539,15 @@ def fillFormat(self, data):
import numeric as _nc
errstate = _nc.seterr(all='ignore')
try:
- special = isnan(data) | isinf(data) | isna(data)
- special[isna(data)] = False
+ special = isnan(data) | isinf(data)
valid = not_equal(data, 0) & ~special
- valid[isna(data)] = False
non_zero = absolute(data.compress(valid))
if len(non_zero) == 0:
max_val = 0.
min_val = 0.
else:
- max_val = maximum.reduce(non_zero, skipna=True)
- min_val = minimum.reduce(non_zero, skipna=True)
+ max_val = maximum.reduce(non_zero)
+ min_val = minimum.reduce(non_zero)
if max_val >= 1.e8:
self.exp_format = True
if not self.suppress_small and (min_val < 0.0001
@@ -594,8 +578,7 @@ def fillFormat(self, data):
if _nc.any(special):
self.max_str_len = max(self.max_str_len,
len(_nan_str),
- len(_inf_str)+1,
- len(_na_str))
+ len(_inf_str)+1)
if self.sign:
format = '%#+'
else:
@@ -609,9 +592,7 @@ def __call__(self, x, strip_zeros=True):
import numeric as _nc
err = _nc.seterr(invalid='ignore')
try:
- if isna(x):
- return self.special_fmt % (str(x).replace('NA', _na_str, 1),)
- elif isnan(x):
+ if isnan(x):
if self.sign:
return self.special_fmt % ('+' + _nan_str,)
else:
@@ -654,8 +635,8 @@ def _digits(x, precision, format):
class IntegerFormat(object):
def __init__(self, data):
try:
- max_str_len = max(len(str(maximum.reduce(data, skipna=True))),
- len(str(minimum.reduce(data, skipna=True))))
+ max_str_len = max(len(str(maximum.reduce(data))),
+ len(str(minimum.reduce(data))))
self.format = '%' + str(max_str_len) + 'd'
except (TypeError, NotImplementedError):
# if reduce(data) fails, this instance will not be called, just
@@ -666,9 +647,7 @@ def __init__(self, data):
pass
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- elif _MININT < x < _MAXINT:
+ if _MININT < x < _MAXINT:
return self.format % x
else:
return "%s" % x
@@ -681,9 +660,7 @@ def __init__(self, precision, sign=False):
self.sign = sign
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- elif isnan(x):
+ if isnan(x):
if self.sign:
return '+' + _nan_str
else:
@@ -711,12 +688,9 @@ def __init__(self, precision):
self.imag_format = LongFloatFormat(precision, sign=True)
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- else:
- r = self.real_format(x.real)
- i = self.imag_format(x.imag)
- return r + i + 'j'
+ r = self.real_format(x.real)
+ i = self.imag_format(x.imag)
+ return r + i + 'j'
class ComplexFormat(object):
@@ -726,17 +700,14 @@ def __init__(self, x, precision, suppress_small):
sign=True)
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
+ r = self.real_format(x.real, strip_zeros=False)
+ i = self.imag_format(x.imag, strip_zeros=False)
+ if not self.imag_format.exp_format:
+ z = i.rstrip('0')
+ i = z + 'j' + ' '*(len(i)-len(z))
else:
- r = self.real_format(x.real, strip_zeros=False)
- i = self.imag_format(x.imag, strip_zeros=False)
- if not self.imag_format.exp_format:
- z = i.rstrip('0')
- i = z + 'j' + ' '*(len(i)-len(z))
- else:
- i = i + 'j'
- return r + i
+ i = i + 'j'
+ return r + i
class DatetimeFormat(object):
def __init__(self, x, unit=None,
@@ -761,13 +732,10 @@ def __init__(self, x, unit=None,
self.casting = casting
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- else:
- return "'%s'" % datetime_as_string(x,
- unit=self.unit,
- timezone=self.timezone,
- casting=self.casting)
+ return "'%s'" % datetime_as_string(x,
+ unit=self.unit,
+ timezone=self.timezone,
+ casting=self.casting)
class TimedeltaFormat(object):
def __init__(self, data):
@@ -778,8 +746,5 @@ def __init__(self, data):
self.format = '%' + str(max_str_len) + 'd'
def __call__(self, x):
- if isna(x):
- return str(x).replace('NA', _na_str, 1)
- else:
- return self.format % x.astype('i8')
+ return self.format % x.astype('i8')
View
2  numpy/core/code_generators/genapi.py
@@ -43,8 +43,6 @@
join('multiarray', 'iterators.c'),
join('multiarray', 'methods.c'),
join('multiarray', 'multiarraymodule.c'),
- join('multiarray', 'na_mask.c'),
- join('multiarray', 'na_object.c'),
join('multiarray', 'nditer_api.c'),
join('multiarray', 'nditer_constr.c'),
join('multiarray', 'nditer_pywrap.c'),
View
37 numpy/core/code_generators/numpy_api.py
@@ -14,12 +14,10 @@
multiarray_global_vars = {
'NPY_NUMUSERTYPES': 7,
- 'Npy_NA': 299,
}
multiarray_global_vars_types = {
'NPY_NUMUSERTYPES': 'int',
- 'Npy_NA': 'PyObject *',
}
multiarray_scalar_bool_values = {
@@ -71,7 +69,6 @@
'PyHalfArrType_Type': 217,
'NpyIter_Type': 218,
# End 1.6 API
- 'NpyNA_Type': 281,
}
#define NPY_NUMUSERTYPES (*(int *)PyArray_API[6])
@@ -320,32 +317,14 @@
'PyArray_ConvertClipmodeSequence': 279,
'PyArray_MatrixProduct2': 280,
# End 1.6 API
- 'NpyIter_GetFirstMaskNAOp': 282,
- 'NpyIter_GetMaskNAIndexArray': 283,
- 'NpyIter_IsFirstVisit': 284,
- 'PyArray_SetBaseObject': 285,
- 'PyArray_HasNASupport': 286,
- 'PyArray_ContainsNA': 287,
- 'PyArray_AllocateMaskNA': 288,
- 'PyArray_CreateSortedStridePerm': 289,
- 'PyArray_AssignZero': 290,
- 'PyArray_AssignOne': 291,
- 'PyArray_AssignNA': 292,
- 'PyArray_AssignMaskNA': 293,
- 'PyArray_AssignRawScalar': 294,
- 'PyArray_AssignArray': 295,
- 'PyArray_ReduceWrapper': 296,
- 'PyArray_RemoveAxesInPlace': 297,
- 'PyArray_DebugPrint': 298,
- 'NpyNA_GetDType': 300,
- 'NpyNA_IsMultiNA': 301,
- 'NpyNA_GetPayload': 302,
- 'NpyNA_FromObject': 303,
- 'NpyNA_FromDTypeAndPayload': 304,
- 'PyArray_AllowNAConverter': 305,
- 'PyArray_OutputAllowNAConverter': 306,
- 'PyArray_FailUnlessWriteable': 307,
- 'PyArray_SetUpdateIfCopyBase': 308,
+ 'NpyIter_IsFirstVisit': 281,
+ 'PyArray_SetBaseObject': 282,
+ 'PyArray_ReduceWrapper': 283,
+ 'PyArray_CreateSortedStridePerm': 284,
+ 'PyArray_RemoveAxesInPlace': 285,
+ 'PyArray_DebugPrint': 286,
+ 'PyArray_FailUnlessWriteable': 287,
+ 'PyArray_SetUpdateIfCopyBase': 288,
}
ufunc_types_api = {
View
90 numpy/core/fromnumeric.py
@@ -1419,7 +1419,7 @@ def clip(a, a_min, a_max, out=None):
return clip(a_min, a_max, out)
-def sum(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def sum(a, axis=None, dtype=None, out=None, keepdims=False):
"""
Sum of array elements over a given axis.
@@ -1449,9 +1449,6 @@ def sum(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
(the shape of `a` with `axis` removed, i.e.,
``numpy.delete(a.shape, axis)``). Its type is preserved. See
`doc.ufuncs` (Section "Output arguments") for more details.
- skipna : bool, optional
- If this is set to True, skips any NA values during summation
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -1510,14 +1507,14 @@ def sum(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
sum = a.sum
except AttributeError:
return _methods._sum(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
- # NOTE: Dropping the skipna and keepdims parameters here...
+ out=out, keepdims=keepdims)
+ # NOTE: Dropping the keepdims parameters here...
return sum(axis=axis, dtype=dtype, out=out)
else:
return _methods._sum(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def product (a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def product (a, axis=None, dtype=None, out=None, keepdims=False):
"""
Return the product of array elements over a given axis.
@@ -1526,10 +1523,10 @@ def product (a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
prod : equivalent function; see for details.
"""
- return um.multiply.reduce(a, axis=axis, dtype=dtype, out=out, skipna=skipna, keepdims=keepdims)
+ return um.multiply.reduce(a, axis=axis, dtype=dtype, out=out, keepdims=keepdims)
-def sometrue(a, axis=None, out=None, skipna=False, keepdims=False):
+def sometrue(a, axis=None, out=None, keepdims=False):
"""
Check whether some values are true.
@@ -1543,11 +1540,11 @@ def sometrue(a, axis=None, out=None, skipna=False, keepdims=False):
arr = asanyarray(a)
try:
- return arr.any(axis=axis, out=out, skipna=skipna, keepdims=keepdims)
+ return arr.any(axis=axis, out=out, keepdims=keepdims)
except TypeError:
return arr.any(axis=axis, out=out)
-def alltrue (a, axis=None, out=None, skipna=False, keepdims=False):
+def alltrue (a, axis=None, out=None, keepdims=False):
"""
Check if all elements of input array are true.
@@ -1559,11 +1556,11 @@ def alltrue (a, axis=None, out=None, skipna=False, keepdims=False):
arr = asanyarray(a)
try:
- return arr.all(axis=axis, out=out, skipna=skipna, keepdims=keepdims)
+ return arr.all(axis=axis, out=out, keepdims=keepdims)
except TypeError:
return arr.all(axis=axis, out=out)
-def any(a, axis=None, out=None, skipna=False, keepdims=False):
+def any(a, axis=None, out=None, keepdims=False):
"""
Test whether any array element along a given axis evaluates to True.
@@ -1589,9 +1586,6 @@ def any(a, axis=None, out=None, skipna=False, keepdims=False):
(e.g., if it is of type float, then it will remain so, returning
1.0 for True and 0.0 for False, regardless of the type of `a`).
See `doc.ufuncs` (Section "Output arguments") for details.
- skipna : bool, optional
- If this is set to True, skips any NA values during summation
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -1642,11 +1636,11 @@ def any(a, axis=None, out=None, skipna=False, keepdims=False):
arr = asanyarray(a)
try:
- return arr.any(axis=axis, out=out, skipna=skipna, keepdims=keepdims)
+ return arr.any(axis=axis, out=out, keepdims=keepdims)
except TypeError:
return arr.any(axis=axis, out=out)
-def all(a, axis=None, out=None, skipna=False, keepdims=False):
+def all(a, axis=None, out=None, keepdims=False):
"""
Test whether all array elements along a given axis evaluate to True.
@@ -1670,9 +1664,6 @@ def all(a, axis=None, out=None, skipna=False, keepdims=False):
type is preserved (e.g., if ``dtype(out)`` is float, the result
will consist of 0.0's and 1.0's). See `doc.ufuncs` (Section
"Output arguments") for more details.
- skipna : bool, optional
- If this is set to True, skips any NA values during summation
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -1718,7 +1709,7 @@ def all(a, axis=None, out=None, skipna=False, keepdims=False):
arr = asanyarray(a)
try:
- return arr.all(axis=axis, out=out, skipna=skipna, keepdims=keepdims)
+ return arr.all(axis=axis, out=out, keepdims=keepdims)
except TypeError:
return arr.all(axis=axis, out=out)
@@ -1853,7 +1844,7 @@ def ptp(a, axis=None, out=None):
return ptp(axis, out)
-def amax(a, axis=None, out=None, skipna=False, keepdims=False):
+def amax(a, axis=None, out=None, keepdims=False):
"""
Return the maximum of an array or maximum along an axis.
@@ -1867,9 +1858,6 @@ def amax(a, axis=None, out=None, skipna=False, keepdims=False):
Alternate output array in which to place the result. Must be of
the same shape and buffer length as the expected output. See
`doc.ufuncs` (Section "Output arguments") for more details.
- skipna : bool, optional
- If this is set to True, skips any NA values during reduction
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -1920,14 +1908,14 @@ def amax(a, axis=None, out=None, skipna=False, keepdims=False):
amax = a.max
except AttributeError:
return _methods._amax(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
- # NOTE: Dropping the skipna and keepdims parameters
+ out=out, keepdims=keepdims)
+ # NOTE: Dropping and keepdims parameter
return amax(axis=axis, out=out)
else:
return _methods._amax(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def amin(a, axis=None, out=None, skipna=False, keepdims=False):
+def amin(a, axis=None, out=None, keepdims=False):
"""
Return the minimum of an array or minimum along an axis.
@@ -1941,9 +1929,6 @@ def amin(a, axis=None, out=None, skipna=False, keepdims=False):
Alternative output array in which to place the result. Must
be of the same shape and buffer length as the expected output.
See `doc.ufuncs` (Section "Output arguments") for more details.
- skipna : bool, optional
- If this is set to True, skips any NA values during reduction
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -1994,12 +1979,12 @@ def amin(a, axis=None, out=None, skipna=False, keepdims=False):
amin = a.min
except AttributeError:
return _methods._amin(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
- # NOTE: Dropping the skipna and keepdims parameters
+ out=out, keepdims=keepdims)
+ # NOTE: Dropping the keepdims parameter
return amin(axis=axis, out=out)
else:
return _methods._amin(a, axis=axis,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
def alen(a):
"""
@@ -2034,7 +2019,7 @@ def alen(a):
return len(array(a,ndmin=1))
-def prod(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def prod(a, axis=None, dtype=None, out=None, keepdims=False):
"""
Return the product of array elements over a given axis.
@@ -2062,9 +2047,6 @@ def prod(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
Alternative output array in which to place the result. It must have
the same shape as the expected output, but the type of the
output values will be cast if necessary.
- skipna : bool, optional
- If this is set to True, skips any NA values during reduction
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -2127,11 +2109,11 @@ def prod(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
prod = a.prod
except AttributeError:
return _methods._prod(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
return prod(axis=axis, dtype=dtype, out=out)
else:
return _methods._prod(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
def cumprod(a, axis=None, dtype=None, out=None):
"""
@@ -2420,7 +2402,7 @@ def round_(a, decimals=0, out=None):
return round(decimals, out)
-def mean(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
+def mean(a, axis=None, dtype=None, out=None, keepdims=False):
"""
Compute the arithmetic mean along the specified axis.
@@ -2445,9 +2427,6 @@ def mean(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
is ``None``; if provided, it must have the same shape as the
expected output, but the type will be cast if necessary.
See `doc.ufuncs` for details.
- skipna : bool, optional
- If this is set to True, skips any NA values during calculation
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
@@ -2506,11 +2485,10 @@ def mean(a, axis=None, dtype=None, out=None, skipna=False, keepdims=False):
pass
return _methods._mean(a, axis=axis, dtype=dtype,
- out=out, skipna=skipna, keepdims=keepdims)
+ out=out, keepdims=keepdims)
-def std(a, axis=None, dtype=None, out=None, ddof=0,
- skipna=False, keepdims=False):
+def std(a, axis=None, dtype=None, out=None, ddof=0, keepdims=False):
"""
Compute the standard deviation along the specified axis.
@@ -2537,9 +2515,6 @@ def std(a, axis=None, dtype=None, out=None, ddof=0,
Means Delta Degrees of Freedom. The divisor used in calculations
is ``N - ddof``, where ``N`` represents the number of elements.
By default `ddof` is zero.
- skipna : bool, optional
- If this is set to True, skips any NA values during calculation
- instead of propagating them.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,