[SPARK-46819][CORE] Move error categories and states into JSON #44863
Conversation
|
@MaxGekk - Before I continue down this path, do you like where this is going? Asking you per @itholic's comment on SPARK-46810. |
|
also cc @cloud-fan |
nchammas
changed the title
|
@MaxGekk - The naming choices I made here are based on my proposal in SPARK-46810. Are you OK with what I wrote there? (If yes, could you chime in on the ticket just for the record, please?) |
|
No rush at all. (Didn't mean to nag. Dunno if my comment came off that way.) |
|
Thanks for doing this. Adding a row to this table is royal pain in the behind. This will make it easier. |
There was a problem hiding this comment.
@nchammas I wonder did you generate error-states.json automatically by a script?
common/utils/src/main/scala/org/apache/spark/ErrorClassesJSONReader.scala
Outdated
Show resolved
Hide resolved
common/utils/src/main/scala/org/apache/spark/ErrorClassesJSONReader.scala
Outdated
Show resolved
Hide resolved
common/utils/src/main/scala/org/apache/spark/ErrorClassesJSONReader.scala
Outdated
Show resolved
Hide resolved
A little scripting and a little IDE kung-fu. |
nchammas
force-pushed
the
SPARK-46819-error-data-automation
branch
from
a6d1ecf
to
54a1a7e
Compare
| - Redshift. | ||
|
|
||
| <!-- SQLSTATE table start --> | ||
| |SQLSTATE |Class|Condition |Subclass|Subcondition |Origin |Standard|Used By | |
There was a problem hiding this comment.
This is not public doc, correct?
There was a problem hiding this comment.
Correct, not public. I will be opening separate PRs to update our public facing documentation on error classes.
|
The failure Run / Run Spark on Kubernetes Integration test is not related to the changes, I do believe. |
What changes were proposed in this pull request?
Move the error categories and states out of the Markdown table in the error README and into standalone JSON files.
I am using the terms "error category" and "error state" as defined in SPARK-46810.
Why are the changes needed?
The data captured in Markdown is not automation-friendly. In future work, I intend to automate the generation of sql-error-conditions-sqlstates.md using these new JSON files.
Additionally, there are many instances of duplicate keys and inconsistent descriptions for the error categories, which this change resolves by moving the data into JSON. This also enables IDE support for basic validation.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
The updated unit tests ensure the following:
42) and states (e.g.42K01) have the correct length and format.Was this patch authored or co-authored using generative AI tooling?
No.