[SPARK-43962][SQL] Improve error messages: CANNOT_DECODE_URL, CANNOT_MERGE_INCOMPATIBLE_DATA_TYPE, CANNOT_PARSE_DECIMAL, CANNOT_READ_FILE_FOOTER, CANNOT_RECOGNIZE_HIVE_TYPE.
#41455
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…MERGE_INCOMPATIBLE_DATA_TYPE, CANNOT_PARSE_DECIMAL, CANNOT_READ_FILE_FOOTER, CANNOT_RECOGNIZE_HIVE_TYPE.
|
cc @MaxGekk @srielau @cloud-fan Please review this when you find some time 🙏 |
MaxGekk
approved these changes
Jun 5, 2023
|
There should be no dependency from the error messages in tests. @itholic Correct? |
|
@itholic Could you re-trigger CI, please. |
|
Yes, that's correct. The existing tests should not fail since I didn't modify the error message parameters. If they do fail, it means they are not properly testing and we need to fix the tests accordingly, possibly using methods like |
|
+1, LGTM. Merging to master. |
czxm
pushed a commit
to czxm/spark
that referenced
this pull request
Jun 12, 2023
…OT_MERGE_INCOMPATIBLE_DATA_TYPE`, `CANNOT_PARSE_DECIMAL`, `CANNOT_READ_FILE_FOOTER`, `CANNOT_RECOGNIZE_HIVE_TYPE` ### What changes were proposed in this pull request? This PR proposes to improve error messages for `CANNOT_DECODE_URL`, `CANNOT_MERGE_INCOMPATIBLE_DATA_TYPE`, `CANNOT_PARSE_DECIMAL`, `CANNOT_READ_FILE_FOOTER`, `CANNOT_RECOGNIZE_HIVE_TYPE`. **NOTE:** This PR is an experimental work that utilizes LLM to enhance error messages. The script was created using the `openai` Python library from OpenAI, and minimal review was conducted by author after executing the script. The five improved error messages were selected by the author. ### Why are the changes needed? For improving errors to make them more actionable and usable. ### Does this PR introduce _any_ user-facing change? No API changes, only error message improvement. ### How was this patch tested? The existing CI should pass. Closes apache#41455 from itholic/emi_1-5. Authored-by: itholic <haejoon.lee@databricks.com> Signed-off-by: Max Gekk <max.gekk@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR proposes to improve error messages for
CANNOT_DECODE_URL,CANNOT_MERGE_INCOMPATIBLE_DATA_TYPE,CANNOT_PARSE_DECIMAL,CANNOT_READ_FILE_FOOTER,CANNOT_RECOGNIZE_HIVE_TYPE.NOTE: This PR is an experimental work that utilizes LLM to enhance error messages. The script was created using the
openaiPython library from OpenAI, and minimal review was conducted by author after executing the script. The five improved error messages were selected by the author.Why are the changes needed?
For improving errors to make them more actionable and usable.
Does this PR introduce any user-facing change?
No API changes, only error message improvement.
How was this patch tested?
The existing CI should pass.