Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Fix PySpark error when doing alias after selection #2127

Merged
merged 4 commits into from
Mar 24, 2020
Merged

BUG: Fix PySpark error when doing alias after selection #2127

merged 4 commits into from
Mar 24, 2020

Conversation

icexelloss
Copy link
Contributor

This PR fixes the bug where PySpark backend is enable to handle this express correctly:

table = ...
table = table[[col]]
table = table.mutate(col2=table[col])

@icexelloss
Copy link
Contributor Author

cc @jreback

@icexelloss icexelloss changed the title [Bug][PySpark] Fix the compiler error when doing alias after selection BUG: Fix the compiler error when doing alias after selection Mar 11, 2020
@icexelloss icexelloss changed the title BUG: Fix the compiler error when doing alias after selection BUG: Fix the PySpark compiler error when doing alias after selection Mar 11, 2020
@icexelloss icexelloss added the bug Incorrect behavior inside of ibis label Mar 11, 2020
@icexelloss icexelloss changed the title BUG: Fix the PySpark compiler error when doing alias after selection BUG: Fix PySpark error when doing alias after selection Mar 11, 2020
jreback
jreback suggested changes Mar 12, 2020
View reviewed changes
ibis/pyspark/compiler.py Show resolved Hide resolved
@jreback jreback added this to the Next Bugfix Release milestone Mar 14, 2020
jreback
jreback suggested changes Mar 14, 2020
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments, ping on green

ibis/pyspark/compiler.py Show resolved Hide resolved
ibis/pyspark/tests/test_basic.py Show resolved Hide resolved
@icexelloss icexelloss mentioned this pull request Mar 16, 2020
Closed
@icexelloss icexelloss force-pushed the pyspark-alias-after-selection-bug branch from 39f33ea to 7d5e691 Compare March 16, 2020 15:21
@icexelloss
Copy link
Contributor Author

@jreback Should be good to merge.

@icexelloss icexelloss force-pushed the pyspark-alias-after-selection-bug branch from 39e6adf to 2d03dd1 Compare March 23, 2020 14:26
@jreback jreback merged commit b0a2df5 into ibis-project:master Mar 24, 2020
@jreback
Copy link
Contributor

jreback commented Mar 24, 2020

thanks @icexelloss

@icexelloss
Copy link
Contributor Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

Assignees
No one assigned
Labels
bug Incorrect behavior inside of ibis
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@icexelloss @jreback