New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-20536][SQL] Extend ColumnName to create StructFields with explicit nullable #20832
Conversation
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends Column(name) { | |||
*/ | |||
def boolean: StructField = StructField(name, BooleanType) | |||
|
|||
/** | |||
* Creates a new `StructField` of type boolean. | |||
* @since 2.3.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2.4.0
I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
/** | ||
* Creates a new `StructField` of type byte. | ||
* @since 1.3.0 | ||
*/ | ||
def byte: StructField = StructField(name, ByteType) | ||
|
||
/** | ||
* Creates a new `StructField` of type byte. | ||
* @since 2.3.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2.4.0
and in the other places too (unless they patch 2.3.0 and becomes 2.3.1).
Can one of the admins verify this patch? |
* Creates a new `StructField` of type boolean. | ||
* @since 2.4.0 | ||
*/ | ||
def boolean(nullable: Boolean): StructField = StructField(name, BooleanType, nullable) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The NULL hints are not enforced. Thus, it is kind of risky to expose this to end users since it could generate a wrong result. We plan to ignore the user-specified NULL hints in the upcoming release.
@efimpoberezkin Could you please close this PR? |
What changes were proposed in this pull request?
Extended ColumnName with methods that create StructFields with explicit nullable property
How was this patch tested?
Existing tests