Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove hard pandas dependency for melt by using methodcaller #4719

Merged
merged 4 commits into from Apr 22, 2019

Conversation

Projects
None yet
3 participants
@beckernick
Copy link
Contributor

commented Apr 19, 2019

Summary of Changes

  • This PR updates the melt function to use M.melt instead of pd.melt, removing the hard coding of dask's melt function to pandas.

  • [ X ] Tests added / passed

    • ====================================== 5358 passed, 472 skipped, 23 xfailed, 6 xpassed, 149 warnings in 793.61 seconds ======================================
  • [ X ] Passes flake8 dask

This closes #4718 .

Show resolved Hide resolved dask/dataframe/reshape.py Outdated

beckernick added some commits Apr 19, 2019

@jcrist

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

Looks like pandas 0.19 doesn't have melt as a dataframe method. Perhaps we should go forward with #4580 and bump the versions now.

@beckernick

This comment has been minimized.

Copy link
Contributor Author

commented Apr 19, 2019

Melt test is failing on travis-CI due to pandas 0.19.2. Melt was introduced as a DataFrame method in pandas version 0.20.0. It passes appveyor due to that build having pandas 0.20.3.

I'm able to pass locally in the following environment, which was created with a fresh python3 virtual environment followed by pip install -e ".[complete]" and pip install pytest:

(dask-dev) nickbecker (feature/melt-generic-methodcaller *) tests $ pip freeze
atomicwrites==1.3.0
attrs==19.1.0
Click==7.0
cloudpickle==0.8.1
-e git+https://github.com/beckernick/dask.git@83d9854962b444c4c722703bd4b38aa69443f9ed#egg=dask
distributed==1.27.0
HeapDict==1.0.0
locket==0.2.0
more-itertools==7.0.0
msgpack==0.6.1
numpy==1.16.2
pandas==0.24.2
partd==0.3.10
pluggy==0.9.0
psutil==5.6.1
py==1.8.0
pytest==4.4.1
python-dateutil==2.8.0
pytz==2019.1
PyYAML==5.1
six==1.12.0
sortedcontainers==2.1.0
tblib==1.3.2
toolz==0.9.0
tornado==6.0.2
zict==0.1.4

Dask's setup.py seems to indicate that pandas >= 0.19.0 should work with dask. Is maintaining 0.19 compatibility necessary?

@beckernick

This comment has been minimized.

Copy link
Contributor Author

commented Apr 19, 2019

Looks like pandas 0.19 doesn't have melt as a dataframe method. Perhaps we should go forward with #4580 and bump the versions now.

You beat me to the punch!

@jcrist

This comment has been minimized.

Copy link
Member

commented Apr 22, 2019

After the dependency bump everything looks good here. Thanks, merging.

@jcrist jcrist merged commit 13ed28d into dask:master Apr 22, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

asmith26 added a commit to asmith26/dask that referenced this pull request Apr 22, 2019

Remove hard pandas dependency for melt by using methodcaller (dask#4719)
* remove hard pandas dependency for melt by using methodcaller

* move import to the top of the file

* whitespace line

jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this pull request May 14, 2019

Remove hard pandas dependency for melt by using methodcaller (dask#4719)
* remove hard pandas dependency for melt by using methodcaller

* move import to the top of the file

* whitespace line

Thomas-Z added a commit to Thomas-Z/dask that referenced this pull request May 17, 2019

Remove hard pandas dependency for melt by using methodcaller (dask#4719)
* remove hard pandas dependency for melt by using methodcaller

* move import to the top of the file

* whitespace line
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.