-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-8766] support non-ascii character in column names #7165
Conversation
Merged build triggered. |
Merged build started. |
Test build #36299 has started for PR 7165 at commit |
Test build #36299 has finished for PR 7165 at commit
|
Merged build finished. Test FAILed. |
Merged build triggered. |
Merged build started. |
Test build #36306 has started for PR 7165 at commit |
Test build #36306 has finished for PR 7165 at commit
|
Merged build finished. Test FAILed. |
Merged build triggered. |
Merged build started. |
Test build #36313 has started for PR 7165 at commit |
Test build #36313 has finished for PR 7165 at commit
|
Merged build finished. Test PASSed. |
cc @rxin |
LGTM |
Use UTF-8 to encode the name of column in Python 2, or it may failed to encode with default encoding ('ascii'). This PR also fix a bug when there is Java exception without error message. Author: Davies Liu <davies@databricks.com> Closes #7165 from davies/non_ascii and squashes the following commits: 02cb61a [Davies Liu] fix tests 3b09d31 [Davies Liu] add encoding in header 867754a [Davies Liu] support non-ascii character in column names (cherry picked from commit f958f27) Signed-off-by: Davies Liu <davies@databricks.com> Conflicts: python/pyspark/sql/utils.py
Use UTF-8 to encode the name of column in Python 2, or it may failed to encode with default encoding ('ascii').
This PR also fix a bug when there is Java exception without error message.