-
Notifications
You must be signed in to change notification settings - Fork 28.7k
[SPARK-42672][PYTHON][DOCS] Document error class list #40282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Just created ticket for SQL side: SPARK-42706 FYI. |
Documentation for SQL side is get merged from #40336. Note that Python side are simpler compared to SQL side because we do not have SQLSTATE, and there is currently no main error class with sub-error classes. Also, the overall volume of errors is not as high as in SQL documents. |
Reminder for @HyukjinKwon @srielau @MaxGekk for error class document for PySpark. |
Error conditions in PySpark | ||
=========================== | ||
|
||
This is a list of common, named error conditions returned by PySpark which are defined at `error_classes.py <https://github.com/apache/spark/blob/master/python/pyspark/errors/error_classes.py>`_. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just wonder how error classes
become error conditions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually yeah. Is this the proper term we use for error framework?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I referred this term from Error conditions in Databricks because they also support error classes in a similar form.
From my guess, it seems that "error conditions" are used as a concept that also includes error classes and their corresponding parameterized messages.
@srielau do you happen to have any context for this naming?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe do we just update to use term "error classes" instead of "error conditions" ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, let's use the term "error classes", at least for consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. Just applied changes.
|
||
When writing PySpark errors, developers must use an error class from the list. If an appropriate error class is not available, add a new one into the list. For more information, please refer to `Contributing Error and Exception <https://spark.apache.org/docs/latest/api/python/development/contributing.html#contributing-error-and-exception>`_. | ||
|
||
+------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be great if you add a comment here that how you generated this table.
I am merging this to proceed but would be great if you can make a followup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. Will automate the doc generation as follow-up in SPARK-43047.
Merged to master. |
What changes were proposed in this pull request?
This PR proposes to document the PySpark error class list into a part of Development page.
Why are the changes needed?
To make it easier for developers to find error classes.
Does this PR introduce any user-facing change?
No, it's documentation,
How was this patch tested?
Manually build docs by running
make clean html