New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Foreign table joins #1282

Merged
merged 5 commits into from Dec 4, 2015

Conversation

Projects
None yet
3 participants
@cpcloud
Member

cpcloud commented Oct 29, 2015

Right now, this is very simple and assumes that everything fits
in memory.

We convert both pieces of data to DataFrames and then execute the
join.

@jreback

This comment has been minimized.

Contributor

jreback commented Oct 29, 2015

perfect example of where dask would be useful for ooc as a back-end here, cc @mrocklin

@cpcloud cpcloud added the enhancement label Oct 29, 2015

@cpcloud cpcloud added this to the 0.9.0 milestone Oct 29, 2015

@cpcloud cpcloud self-assigned this Oct 29, 2015

@cpcloud

This comment has been minimized.

Member

cpcloud commented Oct 29, 2015

hm ... a later version of this could odo everything to dask dataframes instead or use psutil + memory usage to figure out if both frames will fit in memory

@llllllllll

This comment has been minimized.

Member

llllllllll commented Nov 23, 2015

Could you put this in the "experimental features" section of the whatsnew? Also, are you waiting on anything else to merge this?

@cpcloud

This comment has been minimized.

Member

cpcloud commented Dec 4, 2015

merging on pass

llllllllll added a commit that referenced this pull request Dec 4, 2015

@llllllllll llllllllll merged commit 65e95f9 into blaze:master Dec 4, 2015

@cpcloud

This comment has been minimized.

Member

cpcloud commented Dec 4, 2015

thx for merging

@cpcloud cpcloud deleted the cpcloud:multi-table-join branch Dec 4, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment