Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-43011][SQL] array_insert should fail with 0 index #40641

Closed

Conversation

zhengruifeng
Copy link
Contributor

@zhengruifeng zhengruifeng commented Apr 3, 2023

What changes were proposed in this pull request?

Make array_insert fail when input index pos is zero.

Why are the changes needed?

see #40563 (comment)

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

updated UT

"message" : [
"The index 0 is invalid. An index shall be either < 0 or > 0 (the first element has index 1)."
],
"sqlState" : "22003"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just reuse the sqlState of ELEMENT_AT_BY_INDEX_ZERO, is it fine? @itholic

Copy link
Contributor

@itholic itholic Apr 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if a particular SQL state is appropriate for representing a specific error, we can use the same SQL state in multiple error classes.

But on my second thought, maybe can we consolidate ELEMENT_AT_BY_INDEX_ZERO and ARRAY_INSERT_BY_INDEX_ZERO into single error class if they have exactly the same error message??

For example:

  "ZERO_INDEX_ERROR" : {
    "message" : [
      "The index 0 is invalid. An index shall be either < 0 or > 0 (the first element has index 1)."
    ],
    "sqlState" : "22003"
  },

Although I'm not 100% sure if the name ZERO_INDEX_ERROR is proper to cover the both cases. WDYT @srielau @MaxGekk ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about INVALID_INDEX_OF_ZERO?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. We need to start hardening the error classes, though. They are documented API.

@zhengruifeng
Copy link
Contributor Author

cc @cloud-fan @itholic

@github-actions github-actions bot added the DOCS label Apr 3, 2023
@@ -536,7 +536,7 @@
],
"sqlState" : "23505"
},
"ELEMENT_AT_BY_INDEX_ZERO" : {
"INVALID_INDEX_OF_ZERO" : {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Highly likely, it should be moved since error classes must be sorted in the file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, will fix it soon

@@ -289,7 +289,7 @@ Duplicate map key `<key>` was found, please check the input data. If you want to

Found duplicate keys `<keyColumn>`.

### ELEMENT_AT_BY_INDEX_ZERO
### INVALID_INDEX_OF_ZERO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move it according to alphabetical sorting, please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks

@zhengruifeng
Copy link
Contributor Author

@cloud-fan @MaxGekk would you mind taking another look?

@MaxGekk
Copy link
Member

MaxGekk commented Apr 4, 2023

+1, LGTM. Merging to master/3.4.
Thank you, @zhengruifeng and @cloud-fan @itholic for review.

@MaxGekk MaxGekk closed this in 3e9574c Apr 4, 2023
MaxGekk pushed a commit that referenced this pull request Apr 4, 2023
### What changes were proposed in this pull request?
Make `array_insert` fail when input index `pos` is zero.

### Why are the changes needed?
see #40563 (comment)

### Does this PR introduce _any_ user-facing change?
Yes

### How was this patch tested?
updated UT

Closes #40641 from zhengruifeng/sql_array_insert_fails_zero.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
(cherry picked from commit 3e9574c)
Signed-off-by: Max Gekk <max.gekk@gmail.com>
@zhengruifeng zhengruifeng deleted the sql_array_insert_fails_zero branch April 4, 2023 08:23
snmvaughan pushed a commit to snmvaughan/spark that referenced this pull request Jun 20, 2023
### What changes were proposed in this pull request?
Make `array_insert` fail when input index `pos` is zero.

### Why are the changes needed?
see apache#40563 (comment)

### Does this PR introduce _any_ user-facing change?
Yes

### How was this patch tested?
updated UT

Closes apache#40641 from zhengruifeng/sql_array_insert_fails_zero.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
(cherry picked from commit 3e9574c)
Signed-off-by: Max Gekk <max.gekk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants