[SPARK-41406][SQL] Refactor error message for NUM_COLUMNS_MISMATCH to make it more generic
#38937
Conversation
|
Can one of the admins verify this patch? |
|
cc @MaxGekk |
| @@ -230,10 +230,9 @@ org.apache.spark.sql.AnalysisException | |||
| { | |||
| "errorClass" : "NUM_COLUMNS_MISMATCH", | |||
| "messageParameters" : { | |||
| "invalidNumColumns" : "2", | |||
| "invalidOrdinalNum" : "second", | |||
| "leftNumCols" : "1", | |||
| "operator" : "EXCEPTALL", | |||
There was a problem hiding this comment.
hmm, could you output it as EXCEPT ALL
| @@ -932,7 +932,7 @@ | |||
| }, | |||
| "NUM_COLUMNS_MISMATCH" : { | |||
| "message" : [ | |||
| "<operator> can only be performed on tables with the same number of columns, but the first table has <refNumColumns> columns and the <invalidOrdinalNum> table has <invalidNumColumns> columns." | |||
| "<operator> expects matching number of columns. But the left side (target) has <leftNumCols> while the right side (source) has <rightNumCols>." | |||
There was a problem hiding this comment.
expects matching number of columns -> expect matching numbers of columns
There was a problem hiding this comment.
What is an 'operator' here and can the left always be called the target?
There was a problem hiding this comment.
I'm thinking of the operator being UNION (hence left and right).
So:
SET
INSERT
UNION
INTERSECT
EXCEPT
IN (?) -- this may go through struct logic not sure)
There was a problem hiding this comment.
I would avoid words like left/right and reference/target, and just slightly improve the existing error template by replacing the word table to let's say input:
"<operator> can only be performed on inputs with the same number of columns, but the first input has <firstNumColumns> columns and the <invalidOrdinalNum> input has <invalidNumColumns> columns."
There was a problem hiding this comment.
@panbingkun Could you change the error message, please.
…to make it more generic
…to make it more generic
…to make it more generic
…to make it more generic
panbingkun
force-pushed
the
SPARK-41406
branch
from
fa9fdff
to
d41d2e1
Compare
| @@ -1011,7 +1012,7 @@ class DataFrameSetOperationsSuite extends QueryTest with SharedSparkSession { | |||
| val errMsg = intercept[AnalysisException] { | |||
| df1.unionByName(df2) | |||
| }.getMessage | |||
| assert(errMsg.contains("Union can only be performed on tables with" + | |||
| assert(errMsg.contains("UNION can only be performed on tables with" + | |||
There was a problem hiding this comment.
Is it another error class, right?
|
+1, LGTM. All GAs passed. Merging to master. |
…to make it more generic ### What changes were proposed in this pull request? The pr aims to refactor error message for `NUM_COLUMNS_MISMATCH` to make it more generic. ### Why are the changes needed? The changes improve the error framework. ### Does this PR introduce _any_ user-facing change? Yes. ### How was this patch tested? Update existed UT. Pass GA. Closes apache#38937 from panbingkun/SPARK-41406. Authored-by: panbingkun <pbk1982@gmail.com> Signed-off-by: Max Gekk <max.gekk@gmail.com>
What changes were proposed in this pull request?
The pr aims to refactor error message for
NUM_COLUMNS_MISMATCHto make it more generic.Why are the changes needed?
The changes improve the error framework.
Does this PR introduce any user-facing change?
Yes.
How was this patch tested?
Update existed UT.
Pass GA.