Skip to content

Commit

Permalink
BUG: dropna() on single column timezone-aware values (pandas-dev#13407)
Browse files Browse the repository at this point in the history
  • Loading branch information
JQGoh committed Mar 23, 2018
1 parent 01882ba commit cbe3c81
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 2 deletions.
3 changes: 2 additions & 1 deletion doc/source/whatsnew/v0.23.0.txt
Expand Up @@ -902,11 +902,12 @@ Timezones
- :func:`Timestamp.replace` will now handle Daylight Savings transitions gracefully (:issue:`18319`)
- Bug in tz-aware :class:`DatetimeIndex` where addition/subtraction with a :class:`TimedeltaIndex` or array with ``dtype='timedelta64[ns]'`` was incorrect (:issue:`17558`)
- Bug in :func:`DatetimeIndex.insert` where inserting ``NaT`` into a timezone-aware index incorrectly raised (:issue:`16357`)
- Bug in the :class:`DataFrame` constructor, where tz-aware Datetimeindex and a given column name will result in an empty ``DataFrame`` (:issue:`19157`)
- Bug in :class:`DataFrame` constructor, where tz-aware Datetimeindex and a given column name will result in an empty ``DataFrame`` (:issue:`19157`)
- Bug in :func:`Timestamp.tz_localize` where localizing a timestamp near the minimum or maximum valid values could overflow and return a timestamp with an incorrect nanosecond value (:issue:`12677`)
- Bug when iterating over :class:`DatetimeIndex` that was localized with fixed timezone offset that rounded nanosecond precision to microseconds (:issue:`19603`)
- Bug in :func:`DataFrame.diff` that raised an ``IndexError`` with tz-aware values (:issue:`18578`)
- Bug in :func:`melt` that converted tz-aware dtypes to tz-naive (:issue:`15785`)
- Bug in :func:`Dataframe.count` that raised an ``ValueError`` if .dropna() method is invoked for single column timezone-aware values. (:issue:`13407`)

Offsets
^^^^^^^
Expand Down
4 changes: 3 additions & 1 deletion pandas/core/frame.py
Expand Up @@ -6487,7 +6487,9 @@ def count(self, axis=0, level=None, numeric_only=False):
# column frames with an extension array
result = notna(frame).sum(axis=axis)
else:
counts = notna(frame.values).sum(axis=axis)
# GH13407
series_counts = notna(frame).sum(axis=axis)
counts = series_counts.values
result = Series(counts, index=frame._get_agg_axis(axis))

return result.astype('int64')
Expand Down
30 changes: 30 additions & 0 deletions pandas/tests/frame/test_missing.py
Expand Up @@ -8,6 +8,9 @@
from numpy import nan, random
import numpy as np

import datetime
import dateutil

from pandas.compat import lrange
from pandas import (DataFrame, Series, Timestamp,
date_range, Categorical)
Expand Down Expand Up @@ -183,6 +186,33 @@ def test_dropna_multiple_axes(self):
inp.dropna(how='all', axis=(0, 1), inplace=True)
assert_frame_equal(inp, expected)

def test_dropna_tz_aware_datetime(self):
# GH13407
# Example reported by GH13407
df = DataFrame()
df['Time'] = [datetime.datetime(
2015, 1, 1, tzinfo=dateutil.tz.tzutc())]
result = df.dropna(axis=0)
expected = DataFrame({'Time': [datetime.datetime(
2015, 1, 1, tzinfo=dateutil.tz.tzutc())]})
assert_frame_equal(result, expected)

# Ex2
df2 = DataFrame({'Time': [datetime.datetime(
2015, 1, 1, tzinfo=dateutil.tz.tzutc()),
None,
np.nan,
datetime.datetime(2015, 2, 2,
tzinfo=dateutil.tz.tzutc())]})
result2 = df2.dropna(axis=0)
expected2 = DataFrame([datetime.datetime(
2015, 1, 1, tzinfo=dateutil.tz.tzutc()),
datetime.datetime(2015, 2, 2,
tzinfo=dateutil.tz.tzutc())],
columns=['Time'],
index=[0, 3])
assert_frame_equal(result2, expected2)

def test_fillna(self):
tf = self.tsframe
tf.loc[tf.index[:5], 'A'] = nan
Expand Down

0 comments on commit cbe3c81

Please sign in to comment.