Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: std for dt64 dtype #37436

Merged
merged 6 commits into from
Oct 30, 2020
Merged

ENH: std for dt64 dtype #37436

merged 6 commits into from
Oct 30, 2020

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@jreback jreback added ExtensionArray Extending pandas with custom dtypes or arrays. Numeric Operations Arithmetic, Comparison, and Logical operations Datetime Datetime data dtype labels Oct 27, 2020
@jreback jreback added this to the 1.2 milestone Oct 27, 2020
@@ -785,6 +784,9 @@ def nanstd(values, axis=None, skipna=True, ddof=1, mask=None):
>>> nanops.nanstd(s)
1.0
"""
if values.dtype == "M8[ns]":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be done in _get_values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is specific to std

@@ -290,8 +290,18 @@ def test_sum_2d_skipna_false(self):
)._values
tm.assert_timedelta_array_equal(result, expected)

def test_std(self):
tdi = pd.TimedeltaIndex(["0H", "4H", "NaT", "4H", "0H", "2H"])
# Adding a Timestamp makes this a test for DatetimeArray.std
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be in test_datetimes? yes i read your comment, but still the location is odd

result = nanops.nanstd(np.asarray(arr), skipna=False)
assert result is pd.NaT
if getattr(arr, "tz", None) is None:
result = nanops.nanstd(np.asarray(arr), skipna=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we sure the actual result is correct? e.g. you are just comparing vs the implementation itself. IOW a result which checks a fixed series of datetimes for mean/std would be good.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed these tests needs fleshing out. im still giving thought to where they should live

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, that's fine about location. i am more concerned with if the results are actually correct; reasoning is that they are converted to i8 but need to make sure that these are coming back correct as well. i am not sure this is actually tested.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we have tests with explicit expected for std() with td64 in test_reductions, test_describe, and test_analytics.

@jreback jreback merged commit 927c83c into pandas-dev:master Oct 30, 2020
@jbrockmendel jbrockmendel deleted the enh-dta-std branch October 30, 2020 17:00
kesmit13 pushed a commit to kesmit13/pandas that referenced this pull request Nov 2, 2020
ukarroum pushed a commit to ukarroum/pandas that referenced this pull request Nov 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype ExtensionArray Extending pandas with custom dtypes or arrays. Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants