Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH-19629: Adding numpy nansun/nanmean, etc etc to _cython_table #19670

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
0d39286
Adding numpy nansun/nanmean, etc etc to _cython_table
AaronCritchley Feb 13, 2018
47936c7
Adding in tests, implementing suggested whatsnew entry
AaronCritchley Feb 13, 2018
d2671e6
Fixing flake8 errors
AaronCritchley Feb 13, 2018
29ccb18
PR comments, support for np.nanprod
AaronCritchley Mar 2, 2018
b702747
Merge branch 'master' of git://github.com/pandas-dev/pandas into ENH-…
AaronCritchley Mar 2, 2018
130f767
skipping if nanprod not implemented
AaronCritchley Mar 2, 2018
0e4657a
Checking np version explicitly
AaronCritchley Mar 2, 2018
64c0d93
Using pytest params
AaronCritchley Mar 2, 2018
fdaeaf9
Updating for np 1.12
AaronCritchley Mar 9, 2018
cabb307
Merge branch 'master' of git://github.com/pandas-dev/pandas into ENH-…
AaronCritchley Mar 9, 2018
0c5a2ae
Fixing bad indentation
AaronCritchley Mar 9, 2018
7157161
Moving compat test functions inline to prevent build time issue
AaronCritchley Mar 9, 2018
93e332d
Merge branch 'master' of git://github.com/pandas-dev/pandas into ENH-…
AaronCritchley Mar 22, 2018
8c2a5dd
Making use of fixtures for series and df generation
AaronCritchley Mar 22, 2018
5326d56
Fixing silly formatting, removing external function to compare
AaronCritchley Mar 22, 2018
bdd2917
Merge branch 'master' into ENH-19629-np-nan-funcs-to-cython-map
AaronCritchley Apr 17, 2018
528e12b
Restoring whatsnew to normal after messy merge
AaronCritchley Apr 17, 2018
e88ac10
More whatsnew cleanup
AaronCritchley Apr 17, 2018
40e4dff
Merge branch 'master' of git://github.com/pandas-dev/pandas into ENH-…
AaronCritchley Apr 25, 2018
c9ddaff
Merge branch 'ENH-19629-np-nan-funcs-to-cython-map' of github.com:Aar…
AaronCritchley Apr 25, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1098,6 +1098,7 @@ Numeric
- Bug in :class:`DataFrame` flex arithmetic (e.g. ``df.add(other, fill_value=foo)``) with a ``fill_value`` other than ``None`` failed to raise ``NotImplementedError`` in corner cases where either the frame or ``other`` has length zero (:issue:`19522`)
- Multiplication and division of numeric-dtyped :class:`Index` objects with timedelta-like scalars returns ``TimedeltaIndex`` instead of raising ``TypeError`` (:issue:`19333`)
- Bug where ``NaN`` was returned instead of 0 by :func:`Series.pct_change` and :func:`DataFrame.pct_change` when ``fill_method`` is not ``None`` (:issue:`19873`)
- :meth:`~DataFrame.agg` now correctly handles numpy NaN-aware methods like :meth:`numpy.nansum` (:issue:`19629`)

Strings
^^^^^^^
Expand Down
19 changes: 17 additions & 2 deletions pandas/core/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
from pandas.core import common as com, algorithms
import pandas.core.nanops as nanops
import pandas._libs.lib as lib
from pandas.compat.numpy import function as nv
from pandas.compat.numpy import (function as nv, _np_version_under1p10,
_np_version_under1p12)
from pandas.compat import PYPY
from pandas.util._decorators import (Appender, cache_readonly,
deprecate_kwarg, Substitution)
Expand Down Expand Up @@ -191,17 +192,31 @@ class SelectionMixin(object):
np.all: 'all',
np.any: 'any',
np.sum: 'sum',
np.nansum: 'sum',
np.mean: 'mean',
np.nanmean: 'mean',
np.prod: 'prod',
np.std: 'std',
np.nanstd: 'std',
np.var: 'var',
np.nanvar: 'var',
np.median: 'median',
np.nanmedian: 'median',
np.max: 'max',
np.nanmax: 'max',
np.min: 'min',
np.nanmin: 'min',
np.cumprod: 'cumprod',
np.cumsum: 'cumsum'
np.cumsum: 'cumsum',
}

if not _np_version_under1p10:
_cython_table[np.nanprod] = 'prod'

if not _np_version_under1p12:
_cython_table[np.nancumprod] = 'cumprod'
_cython_table[np.nancumsum] = 'cumsum'

@property
def _selection_name(self):
"""
Expand Down
43 changes: 43 additions & 0 deletions pandas/tests/test_nanops.py
Original file line number Diff line number Diff line change
Expand Up @@ -994,6 +994,49 @@ def prng(self):
return np.random.RandomState(1234)


@pytest.fixture(params=[
pd.Series([1, 2, 3, 4, 5, 6]),
pd.DataFrame([[1, 2, 3], [4, 5, 6]])
])
def nan_test_object(request):
return request.param


@pytest.mark.parametrize("standard, nan_method", [
(np.sum, np.nansum),
(np.mean, np.nanmean),
(np.std, np.nanstd),
(np.var, np.nanvar),
(np.median, np.nanmedian),
(np.max, np.nanmax),
(np.min, np.nanmin)
])
def test_np_nan_functions(standard, nan_method, nan_test_object):
tm.assert_almost_equal(nan_test_object.agg(standard),
nan_test_object.agg(nan_method),
check_exact=True)


@td.skip_if_no("numpy", min_version="1.10.0")
def test_np_nanprod(nan_test_object):
tm.assert_almost_equal(nan_test_object.agg(np.prod),
nan_test_object.agg(np.nanprod),
check_exact=True)


@td.skip_if_no("numpy", min_version="1.12.0")
def test_np_nancumprod(nan_test_object):
# Not using pytest params for methods as will fail at build time
methods = [
(np.cumprod, np.nancumprod),
(np.cumsum, np.nancumsum)
]
for standard, nan_method in methods:
tm.assert_almost_equal(nan_test_object.agg(standard),
nan_test_object.agg(nan_method),
check_exact=True)


def test_use_bottleneck():

if nanops._BOTTLENECK_INSTALLED:
Expand Down