Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BEAM-13939: Restructure Protos to fix namespace conflicts #16961

Merged
merged 8 commits into from
Apr 7, 2022
Merged

BEAM-13939: Restructure Protos to fix namespace conflicts #16961

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c54e923
Updates the Beam model protos to be housed in a namespaced directory …
thempatel Apr 1, 2022
ce48b4a
Updates session.proto to use the new namespaced import
thempatel Apr 1, 2022
ca2a688
Adds a generic proto generated script that can be used to correctly g…
thempatel Apr 1, 2022
0c0db24
Updates the go sdk to use the new genproto script to correctly genera…
thempatel Apr 1, 2022
53ba83d
Regenerates all go sdk proto bindings
thempatel Apr 1, 2022
55968e3
Updates to the python build system. gen_protos.py now supports the ne…
thempatel Apr 1, 2022
cb776c7
Updates to various build, ignore, and linting tooling for python in o…
thempatel Apr 1, 2022
bbf3a61
Updates to the python SDK to use the new import structure created by …
thempatel Apr 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,7 @@ sdks/python/**/*.egg
sdks/python/LICENSE
sdks/python/NOTICE
sdks/python/README.md
sdks/python/apache_beam/portability/api/*pb2*.*
sdks/python/apache_beam/portability/api/*.yaml
sdks/python/apache_beam/portability/api/*
sdks/python/nosetests*.xml
sdks/python/pytest*.xml
sdks/python/postcommit_requirements.txt
Expand Down
2 changes: 1 addition & 1 deletion build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ tasks.rat {
// Default eclipse excludes neglect subprojects

// Proto/grpc generated wrappers
"**/apache_beam/portability/api/*_pb2*.py",
"**/apache_beam/portability/api/**/*_pb2*.py",
"**/go/pkg/beam/**/*.pb.go",

// Ignore go.sum files, which don't permit headers
Expand Down
6 changes: 3 additions & 3 deletions ...xecution/src/main/proto/beam_fn_api.proto → ...m/model/fn_execution/v1/beam_fn_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,12 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/fnexecutio
option java_package = "org.apache.beam.model.fnexecution.v1";
option java_outer_classname = "BeamFnApi";

import "beam_runner_api.proto";
import "endpoints.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/endpoints.proto";
import "org/apache/beam/model/pipeline/v1/metrics.proto";
import "google/protobuf/descriptor.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";
import "metrics.proto";

// A descriptor for connecting to a remote port using the Beam Fn Data API.
// Allows for communication between two environments (for example between the
Expand Down
4 changes: 2 additions & 2 deletions ...n/src/main/proto/beam_provision_api.proto → .../fn_execution/v1/beam_provision_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/fnexecutio
option java_package = "org.apache.beam.model.fnexecution.v1";
option java_outer_classname = "ProvisionApi";

import "beam_runner_api.proto";
import "endpoints.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/endpoints.proto";
import "google/protobuf/struct.proto";

// A service to provide runtime provisioning information to the SDK harness
Expand Down
2 changes: 1 addition & 1 deletion ...src/main/proto/beam_interactive_api.proto → ...interactive/v1/beam_interactive_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ option go_package = "interactive_v1";
option java_package = "org.apache.beam.model.interactive.v1";
option java_outer_classname = "InteractiveApi";

import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "google/protobuf/timestamp.proto";

// The first record. This contains metadata about the stream and how to
Expand Down
2 changes: 1 addition & 1 deletion ...nt/src/main/proto/beam_artifact_api.proto → ...job_management/v1/beam_artifact_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/jobmanagem
option java_package = "org.apache.beam.model.jobmanagement.v1";
option java_outer_classname = "ArtifactApi";

import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";

// A service to retrieve artifacts for use in a Job.
service ArtifactRetrievalService {
Expand Down
2 changes: 1 addition & 1 deletion ...t/src/main/proto/beam_expansion_api.proto → ...ob_management/v1/beam_expansion_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/jobmanagem
option java_package = "org.apache.beam.model.expansion.v1";
option java_outer_classname = "ExpansionApi";

import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";

message ExpansionRequest {
// Set of components needed to interpret the transform, or which
Expand Down
6 changes: 3 additions & 3 deletions ...agement/src/main/proto/beam_job_api.proto → ...odel/job_management/v1/beam_job_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/jobmanagem
option java_package = "org.apache.beam.model.jobmanagement.v1";
option java_outer_classname = "JobApi";

import "beam_runner_api.proto";
import "endpoints.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/endpoints.proto";
import "org/apache/beam/model/pipeline/v1/metrics.proto";
import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";
import "metrics.proto";

// Job Service for running RunnerAPI pipelines
service JobService {
Expand Down
2 changes: 1 addition & 1 deletion ...line/src/main/proto/beam_runner_api.proto → ...m/model/pipeline/v1/beam_runner_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/pipeline_v
option java_package = "org.apache.beam.model.pipeline.v1";
option java_outer_classname = "RunnerApi";

import "endpoints.proto";
import "org/apache/beam/model/pipeline/v1/endpoints.proto";
import "google/protobuf/descriptor.proto";

message BeamConstants {
Expand Down
0 ...l/pipeline/src/main/proto/endpoints.proto → ...he/beam/model/pipeline/v1/endpoints.proto
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions .../src/main/proto/external_transforms.proto → ...del/pipeline/v1/external_transforms.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/pipeline_v
option java_package = "org.apache.beam.model.pipeline.v1";
option java_outer_classname = "ExternalTransforms";

import "schema.proto";
import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/schema.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";

// A configuration payload for an external transform.
// Used as the payload of ExternalTransform as part of an ExpansionRequest.
Expand Down
2 changes: 1 addition & 1 deletion model/pipeline/src/main/proto/metrics.proto → ...ache/beam/model/pipeline/v1/metrics.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ option java_package = "org.apache.beam.model.pipeline.v1";
option java_outer_classname = "MetricsApi";


import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "google/protobuf/descriptor.proto";
import "google/protobuf/timestamp.proto";

Expand Down
0 model/pipeline/src/main/proto/schema.proto → ...pache/beam/model/pipeline/v1/schema.proto
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion .../src/main/proto/standard_window_fns.proto → ...del/pipeline/v1/standard_window_fns.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ option go_package = "github.com/apache/beam/sdks/v2/go/pkg/beam/model/pipeline_v
option java_package = "org.apache.beam.model.pipeline.v1";
option java_outer_classname = "StandardWindowFns";

import "beam_runner_api.proto";
import "org/apache/beam/model/pipeline/v1/beam_runner_api.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";

Expand Down
2 changes: 1 addition & 1 deletion sdks/go/pkg/beam/core/runtime/graphx/v1/gen.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@

package v1

//go:generate protoc -I . v1.proto --go_out=../../../../../../../../../../../
//go:generate ../../../../../../scripts/genproto.sh
Loading