Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace use of explicit type check with ._is_partition_type predicate #4533

Merged
merged 5 commits into from Mar 17, 2019

Conversation

@mrocklin
Copy link
Member

@mrocklin mrocklin commented Feb 27, 2019

  • Tests added / passed
  • Passes flake8 dask
@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Feb 28, 2019

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 7, 2019

I am unable to reproduce locally with a py27 env. I restarted the py36/py27 builds and I'll see what happens

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 7, 2019

Nevermind! Forgot to change branches

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 7, 2019

In py27 pandas fails on hasattr(typ, 'columns'). Note that if you call dir on the object, columns is in the list

Python 2.7.15 |Anaconda, Inc.| (default, Dec 14 2018, 13:10:39)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import pandas as pd

In [2]: hasattr(type(pd.DataFrame()), 'columns')
Out[2]: False

In [3]: 'columns' in dir(type(pd.DataFrame()))
Out[3]: True

In [4]: 'columns' in dir(type(pd.Series()))
Out[4]: False

In [8]: type(pd.DataFrame()).columns
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-2c7dcd4cba03> in <module>()
----> 1 type(pd.DataFrame()).columns

pandas/_libs/src/properties.pyx in pandas._libs.lib.AxisProperty.__get__ (pandas/_libs/lib.c:45152)()

AttributeError: 'NoneType' object has no attribute '_data'

@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Mar 7, 2019

Well that's odd

Python 2.7.15 |Anaconda, Inc.| (default, Dec 14 2018, 13:10:39)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import pandas as pd

In [2]: hasattr(type(pd.DataFrame()), 'columns')
Out[2]: True

@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Mar 7, 2019

In [3]: pd.__version__
Out[3]: u'0.24.1'

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 7, 2019

The test is against pd 0.20.2:

In [10]: pd.__version__
Out[10]: u'0.20.2'

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 7, 2019

In version 0.23 this issue was resolved

@quasiben
Copy link
Member

@quasiben quasiben commented Mar 11, 2019

@mrocklin how would you like to proceed here ? I imagine dropping support for older pandas is not on the table at the moment, is it ?

@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Mar 16, 2019

Fixes #4596

@mrocklin
Copy link
Member Author

@mrocklin mrocklin commented Mar 16, 2019

Merging in 24 hours if there are no comments

@mrocklin mrocklin merged commit 789c7aa into dask:master Mar 17, 2019
2 checks passed
@mrocklin mrocklin deleted the is-partition-type branch Mar 17, 2019
@mrocklin mrocklin mentioned this pull request Mar 19, 2019
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants