-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-32793][SQL] Add raise_error function, adds error message parameter to assert_true #29947
Changes from all commits
92397c2
387c30c
001ff9c
4adcd70
d997562
9facc82
e1cf7bb
cb685c3
40b9ef9
79b1428
c187226
3bab293
df1fc36
0ff60f4
109af99
c683d78
631394b
1bfa51f
d743ebb
923ac46
5f9e5d5
52e16ec
e5ad9e0
e921f66
5b93fd4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -137,6 +137,8 @@ def sha1(col: ColumnOrName) -> Column: ... | |
def sha2(col: ColumnOrName, numBits: int) -> Column: ... | ||
def hash(*cols: ColumnOrName) -> Column: ... | ||
def xxhash64(*cols: ColumnOrName) -> Column: ... | ||
def assert_true(col: ColumnOrName, errMsg: Union[Column, str] = ...): ... | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Two small notes (sorry for being late):
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @karenfeng Could you fix them above in followup? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This might indicate intention here, though technically speaking it's still a def raise_error(errMsg: Union[Column, str]) -> Column: ... is still correct (and literal one). Do you have any thoughts about it @HyukjinKwon? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think doing |
||
def raise_error(errMsg: Union[Column, str]): ... | ||
def concat(*cols: ColumnOrName) -> Column: ... | ||
def concat_ws(sep: str, *cols: ColumnOrName) -> Column: ... | ||
def decode(col: ColumnOrName, charset: str) -> Column: ... | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we throw an exception if
length(errMsg) != 1
? Just in case user does something like this?i.e.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, more checks should be fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In practice we make this check anyway, so it is only a question if we do something about it.