Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame Quantile Broken with Datetime Data #23925

Closed
WillAyd opened this issue Nov 26, 2018 · 1 comment · Fixed by #27436
Closed

DataFrame Quantile Broken with Datetime Data #23925

WillAyd opened this issue Nov 26, 2018 · 1 comment · Fixed by #27436
Assignees
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Bug DataFrame DataFrame data structure Datetime Datetime data dtype
Milestone

Comments

@WillAyd
Copy link
Member

WillAyd commented Nov 26, 2018

This works fine:

In [17]: pd.Series(pd.date_range('1/1/18', periods=5)).quantile()                                                                          
Out[17]: Timestamp('2018-01-03 00:00:00')

But the equivalent method with a DataFrame raises:

In [18]: pd.DataFrame(pd.date_range('1/1/18', periods=5)).quantile()                                                                       
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-18-68ffc067f6f0> in <module>
----> 1 pd.DataFrame(pd.date_range('1/1/18', periods=5)).quantile()

~/clones/pandas/pandas/core/frame.py in quantile(self, q, axis, numeric_only, interpolation)
   7569                                      axis=1,
   7570                                      interpolation=interpolation,
-> 7571                                      transposed=is_transposed)
   7572 
   7573         if result.ndim == 2:

~/clones/pandas/pandas/core/internals/managers.py in quantile(self, **kwargs)
    500 
    501     def quantile(self, **kwargs):
--> 502         return self.reduction('quantile', **kwargs)
    503 
    504     def setitem(self, **kwargs):

~/clones/pandas/pandas/core/internals/managers.py in reduction(self, f, axis, consolidate, transposed, **kwargs)
    473 
    474         # single block
--> 475         values = _concat._concat_compat([b.values for b in blocks])
    476 
    477         # compute the orderings of our original data

~/clones/pandas/pandas/core/dtypes/concat.py in _concat_compat(to_concat, axis)
    172                 to_concat = [x.astype('object') for x in to_concat]
    173 
--> 174     return np.concatenate(to_concat, axis=axis)
    175 
    176 

ValueError: need at least one array to concatenate

Expected Output

The DataFrame should return just as Series does

Output of pd.show_versions()

INSTALLED VERSIONS

commit: 07b0c00
python: 3.6.7.final.0
python-bits: 64
OS: Darwin
OS-release: 18.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.0.dev0+1150.g07b0c003d.dirty
pytest: 4.0.0
pip: 18.1
setuptools: 40.6.2
Cython: 0.29
numpy: 1.15.4
scipy: 1.1.0
pyarrow: 0.11.1
xarray: 0.11.0
IPython: 7.1.1
sphinx: 1.8.2
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.8
feather: None
matplotlib: 3.0.1
openpyxl: 2.5.9
xlrd: 1.1.0
xlwt: 1.2.0
xlsxwriter: 1.1.2
lxml: 4.2.5
bs4: 4.6.3
html5lib: 1.0.1
sqlalchemy: 1.2.14
pymysql: 0.9.2
psycopg2: None
jinja2: 2.10
s3fs: 0.1.6
fastparquet: 0.1.6
pandas_gbq: None
pandas_datareader: None
gcsfs: 0.2.0

@WillAyd WillAyd added Bug Datetime Datetime data dtype Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff DataFrame DataFrame data structure labels Nov 26, 2018
@qwhelan
Copy link
Contributor

qwhelan commented Nov 28, 2018

It appears that DataFrame.quantile() has a default of numeric_only=True, which Series.quantile() does not have:

In [5]: pd.DataFrame(pd.date_range('1/1/18', periods=5)).quantile(numeric_only=False)
Out[5]: 0   2018-01-03
        Name: 0.5, dtype: datetime64[ns]

Looks like this is a duplicate of #18608 - I'd be in favor of resolving by deprecating this flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Bug DataFrame DataFrame data structure Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants