Skip to content

Implement protobuf serialization for Expr::ScalarSubquery(_) | Expr::InSubquery { .. } | Expr::Exists { .. } #4337

@alamb

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Note depends on:

In IOx, when a user issued an unsupported subquery, they got a very confusing message about serialization

❯ select count(*) from foo where exists (select column1 from foo);
Error during planning: Error encoding expr as protobuf: General error: Proto serialization error: Expr::ScalarSubquery(_) | Expr::InSubquery { .. } | Expr::Exists { .. }#6195

This is because the EXISTS predicate could not be sent / serialized. If it could have been, the code in the physical plan would have said "does not support logical exists (SUBQUERY)"

Describe the solution you'd like
I would like to implement the protobuf serialization for such subqueries. This would allow the user to see the real error

❯ select count(*) from foo where exists (select column1 from foo);
NotImplemented("Physical plan does not support logical expression EXISTS (<subquery>)")

https://github.com/apache/arrow-datafusion/blob/d355f69aae2cc951cfd021e5c0b690861ba0c4ac/datafusion/proto/src/to_proto.rs#L850-L853

However in order to do so we would likely have to serialize entire LogicalPlans -- #4338

Describe alternatives you've considered
We could put a specific error in IOx for this error but I would prefer to implement the protobuf serialization because it will stay in sync with DataFusions features (e.g. when datafusion does support this kind of query, we won't have to remember to remove the IOx specific check)

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions