[MINOR][PYTHON][DOCS] Fix types and docstring in DataFrame.toDF#38834
Closed
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
Closed
[MINOR][PYTHON][DOCS] Fix types and docstring in DataFrame.toDF#38834HyukjinKwon wants to merge 1 commit intoapache:masterfrom
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
Conversation
HyukjinKwon
changed the title
Contributor
|
LGTM |
Member
Author
|
Merged to master. |
Contributor
|
late lgtm, thanks |
beliefer
pushed a commit
to beliefer/spark
that referenced
this pull request
Dec 15, 2022
### What changes were proposed in this pull request?
`df.toDF` cannot take `Column`s:
```python
>>> df.toDF(df.id)
```
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../spark/python/pyspark/sql/dataframe.py", line 4606, in toDF
jdf = self._jdf.toDF(self._jseq(cols))
File "/.../spark/python/pyspark/sql/dataframe.py", line 2413, in _jseq
return _to_seq(self.sparkSession._sc, cols, converter)
File "/.../spark/python/pyspark/sql/column.py", line 88, in _to_seq
return sc._jvm.PythonUtils.toSeq(cols)
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1314, in __call__
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1277, in _build_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1264, in _get_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_collections.py", line 511, in convert
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1314, in __call__
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1277, in _build_args
File "/.../forked/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1264, in _get_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_collections.py", line 510, in convert
File "/.../spark/python/pyspark/sql/column.py", line 622, in __iter__
raise TypeError("Column is not iterable")
TypeError: Column is not iterable
```
This PR fixes the type and docstrings to remove the mention about `Column`
### Why are the changes needed?
To provide the correct documentation to the end users.
### Does this PR introduce _any_ user-facing change?
No for the main codes.
Yes for the docs.
### How was this patch tested?
CI in this PR should verify it via Python linters.
Closes apache#38834 from HyukjinKwon/minor-docs-todf.
Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
beliefer
pushed a commit
to beliefer/spark
that referenced
this pull request
Dec 18, 2022
### What changes were proposed in this pull request?
`df.toDF` cannot take `Column`s:
```python
>>> df.toDF(df.id)
```
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../spark/python/pyspark/sql/dataframe.py", line 4606, in toDF
jdf = self._jdf.toDF(self._jseq(cols))
File "/.../spark/python/pyspark/sql/dataframe.py", line 2413, in _jseq
return _to_seq(self.sparkSession._sc, cols, converter)
File "/.../spark/python/pyspark/sql/column.py", line 88, in _to_seq
return sc._jvm.PythonUtils.toSeq(cols)
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1314, in __call__
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1277, in _build_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1264, in _get_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_collections.py", line 511, in convert
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1314, in __call__
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1277, in _build_args
File "/.../forked/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1264, in _get_args
File "/.../spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_collections.py", line 510, in convert
File "/.../spark/python/pyspark/sql/column.py", line 622, in __iter__
raise TypeError("Column is not iterable")
TypeError: Column is not iterable
```
This PR fixes the type and docstrings to remove the mention about `Column`
### Why are the changes needed?
To provide the correct documentation to the end users.
### Does this PR introduce _any_ user-facing change?
No for the main codes.
Yes for the docs.
### How was this patch tested?
CI in this PR should verify it via Python linters.
Closes apache#38834 from HyukjinKwon/minor-docs-todf.
Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
df.toDFcannot takeColumns:This PR fixes the type and docstrings to remove the mention about
ColumnWhy are the changes needed?
To provide the correct documentation to the end users.
Does this PR introduce any user-facing change?
No for the main codes.
Yes for the docs.
How was this patch tested?
CI in this PR should verify it via Python linters.