Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BEAM-13939: Restructure Protos to fix namespace conflicts #16961

Merged
merged 8 commits into from
Apr 7, 2022
Merged

BEAM-13939: Restructure Protos to fix namespace conflicts #16961

merged 8 commits into from
Apr 7, 2022

Commits on Apr 7, 2022

  1. Updates the Beam model protos to be housed in a namespaced directory … 

    …structure. This is so that the proto files require usage of a org/apache/beam/model namespace in their imports and so that the generated files also include this namespace in their source file metadata.
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    c54e923 View commit details
    Browse the repository at this point in the history

  2. Updates session.proto to use the new namespaced import

    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    ce48b4a View commit details
    Browse the repository at this point in the history

  3. Adds a generic proto generated script that can be used to correctly g… 

    …enerate all proto files for the go sdk. This new tool will add any necessary options to the proto compiler and generate the proto files relative to the go sdk root to ensure that the generated files have a namespaced file path in their metadata. If you want to generate a proto file in the go sdk, simply use this script in the go:generate directive, the rest will be taken care of by the script.
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    ca2a688 View commit details
    Browse the repository at this point in the history

  4. Updates the go sdk to use the new genproto script to correctly genera… 

    …te proto bindings. Updates the README for how to generate the model proto bindings into the SDK
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    0c0db24 View commit details
    Browse the repository at this point in the history

  5. Regenerates all go sdk proto bindings

    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    53ba83d View commit details
    Browse the repository at this point in the history

  6. Updates to the python build system. gen_protos.py now supports the ne… 

    …w namespaced structure of the Beam model. It does this by supporting arbitrary directory structures of proto files by calculating and replacing the generated imports with relative imports with the generated source. Additionally, it will generate bindings that allow for imports of the form `from apache_beam.portability.api import beam_runner_api_pb2` so that the SDK is not dependent on the potentially changing structure of the generated bindings within `api`. Imports of the form `from apache_beam.portability.api.org.apache.beam.model import beam_runner_api_pb2` are still supported. setup.py now attempts to generate the proto bindings on invocation since the package structure must exist before the wheel can be created.
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    55968e3 View commit details
    Browse the repository at this point in the history

  7. Updates to various build, ignore, and linting tooling for python in o… 

    …rder to support the new python output structure
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    cb776c7 View commit details
    Browse the repository at this point in the history

  8. Updates to the python SDK to use the new import structure created by … 

    …gen_protos
    @thempatel
    thempatel committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    bbf3a61 View commit details
    Browse the repository at this point in the history