Skip to content

Commit

Permalink
[SPARK-48639][CONNECT][PYTHON] Add Origin to Relation.RelationCommon
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR proposes to add `Origin` (from apache#46789) to `Relation.RelationCommon`

### Why are the changes needed?

To have the common protobuf message to keep the source code info.

### Does this PR introduce _any_ user-facing change?

No. This is not used.

### How was this patch tested?

CI should validate protobuf definition, and exiting tests should pass.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47024 from HyukjinKwon/SPARK-48639.

Lead-authored-by: Hyukjin Kwon <gurwls223@apache.org>
Co-authored-by: Hyukjin Kwon <gurwls223@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon and HyukjinKwon committed Jun 24, 2024
1 parent e459674 commit 09cb592
Show file tree
Hide file tree
Showing 3 changed files with 180 additions and 161 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import "google/protobuf/any.proto";
import "spark/connect/expressions.proto";
import "spark/connect/types.proto";
import "spark/connect/catalog.proto";
import "spark/connect/common.proto";

option java_multiple_files = true;
option java_package = "org.apache.spark.connect.proto";
Expand Down Expand Up @@ -106,13 +107,14 @@ message Unknown {}

// Common metadata of all relations.
message RelationCommon {
// TODO(SPARK-48639): Add origin like Expression.ExpressionCommon

// (Required) Shared relation metadata.
string source_info = 1;
// (Optional) Shared relation metadata.
reserved 1;

// (Optional) A per-client globally unique id for a given connect plan.
optional int64 plan_id = 2;

// (Optional) Keep the information of the origin for this expression such as stacktrace.
Origin origin = 3;
}

// Relation that uses a SQL query to generate the output.
Expand Down
Loading

0 comments on commit 09cb592

Please sign in to comment.