[SPARK-47504][SQL] Resolve AbstractDataType simpleStrings for StringTypeCollated #45694
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HyukjinKwon
reviewed
Mar 26, 2024
...lyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CollationTypeConstraints.scala
Outdated
Show resolved
Hide resolved
mihailom-db
changed the title
cloud-fan
reviewed
Apr 8, 2024
| @@ -30,6 +30,7 @@ abstract class StringTypeCollated extends AbstractDataType { | |||
| * Use StringTypeBinary for expressions supporting only binary collation. | |||
| */ | |||
| case object StringTypeBinary extends StringTypeCollated { | |||
| // TODO (SPARK-47504): Rename this AbstractDataType's simpleString when it is first used | |||
There was a problem hiding this comment.
can we fix it now as we know it should be "string"?
cloud-fan
approved these changes
Apr 8, 2024
|
@cloud-fan Could we merge this in? So some new PR does not conflict with it again. |
|
thanks, merging to master! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Renaming simpleString in StringTypeAnyCollation. This PR should only be merged after #45383 is merged.
Why are the changes needed?
SPARK-47296 introduced a change to fail all unsupported functions. Because of this change expected inputTypes in ExpectsInputTypes had to be changed. This change introduced a change on user side which will print "STRING_ANY_COLLATION" in places where before we printed "STRING" when an error occurred. Concretely if we get an input of Int where StringTypeAnyCollation was expected, we will throw this faulty message for users.
Does this PR introduce any user-facing change?
Yes
How was this patch tested?
Existing tests were changed back to "STRING" notation.
Was this patch authored or co-authored using generative AI tooling?
No.