-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-10417][SQL] Iterating through Column results in infinite loop #8574
Conversation
Don't know much about Python myself but that sounds convincing. CC @davies |
break | ||
self.assertEqual(0, 1) | ||
except TypeError: | ||
self.assertEqual(1, 1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can use assertRaises
to test the exception case.
Test build #1712 has finished for PR 8574 at commit
|
@cloud-fan, I addressed your comments with last commit |
Looks like it's not being retested after the last commit as Jenkins failed to update the status and the dashboard shows that it's still running. Am I right? |
Jenkins, retest this please |
LGTM |
Test build #1714 has finished for PR 8574 at commit
|
Merged into master, thanks! |
pyspark.sql.column.Column
object has__getitem__
method, which makes it iterable for Python. In fact it has__getitem__
to address the case when the column might be a list or dict, for you to be able to access certain element of it in DF API. The ability to iterate over it is just a side effect that might cause confusion for the people getting familiar with Spark DF (as you might iterate this way on Pandas DF for instance)Issue reproduction: