Skip to content

[SPARK-41526][CONNECT][PYTHON] Implement Column.isin#39074

Closed
zhengruifeng wants to merge 1 commit intoapache:masterfrom
zhengruifeng:connect_column_isin
Closed

[SPARK-41526][CONNECT][PYTHON] Implement Column.isin#39074
zhengruifeng wants to merge 1 commit intoapache:masterfrom
zhengruifeng:connect_column_isin

Conversation

@zhengruifeng
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Implement Column.isin

Why are the changes needed?

for API coverage

Does this PR introduce any user-facing change?

yes, new API

How was this patch tested?

added UT

.Product(transformExpression(fun.getArgumentsList.asScala.head))
.toAggregateExpression())

case "in" =>
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we don't intercept in here, it will fail with

    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "[WRONG_NUM_ARGS.WITHOUT_SUGGESTION] Invalid number of arguments for the function `in`. Please, refer to 'https://spark.apache.org/docs/latest/sql-ref-functions.html' for a fix."
        debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:15002 {grpc_message:"[WRONG_NUM_ARGS.WITHOUT_SUGGESTION] Invalid number of arguments for the function `in`. Please, refer to \'https://spark.apache.org/docs/latest/sql-ref-functions.html\' for a fix.", grpc_status:2, created_time:"2022-12-15T17:17:23.380565+08:00"}"
>

This is the second case (the first one is when) that an expression has already been registered in FunctionRegistry, but do not have a constructor matching UnresolvedFunction.

I match them here instead of adding an extra constructor.

@HyukjinKwon
Copy link
Copy Markdown
Member

Merged to master.

@zhengruifeng zhengruifeng deleted the connect_column_isin branch December 16, 2022 01:25
@zhengruifeng
Copy link
Copy Markdown
Contributor Author

@HyukjinKwon THank you for reivews

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants