Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf import path issue #75

Closed
chiperific opened this issue Nov 3, 2023 · 6 comments · Fixed by #79
Closed

Protobuf import path issue #75

chiperific opened this issue Nov 3, 2023 · 6 comments · Fixed by #79

Comments

@chiperific
Copy link

chiperific commented Nov 3, 2023

Warning: Protobuf detected an import path issue while loading generated file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/core_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
Warning: Protobuf detected an import path issue while loading generated file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/schema_service_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
Warning: Protobuf detected an import path issue while loading generated file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/watch_service_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
Warning: Protobuf detected an import path issue while loading generated file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/permission_service_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
Warning: Protobuf detected an import path issue while loading generated file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/experimental_service_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.

Dependencies:

  • rails 7.1.1
  • google-protobuf (3.24.4-aarch64-linux)
  • google-protobuf (3.24.4-arm64-darwin)
  • google-protobuf (3.24.4-x86_64-linux)
  • grpc (1.59.2)
  • grpc-tools (1.59.2)
@chiperific
Copy link
Author

Seems the change is here: https://protobuf.dev/news/2023-04-20/

@vroldanbet
Copy link
Contributor

@chiperific can you provide more context on how you reproduced those warnings? This project uses buf and I don't seem to observe any warnings when using the buf toolchain.

@chiperific
Copy link
Author

chiperific commented Nov 6, 2023

Here's a repo that shows how the warnings appeared in 0.7.0 but aren't in 0.5.0:

https://github.com/chiperific/authzed-protobuf-check

I played with reverting google-protobuf, grpc and grpc-tools to the gem versions in the 0.5.0 release, but the only way I found to remove the error messages was to change authzed.

Locally, when I install google-protobuf, I get google-protobuf (3.24.4-arm64-darwin), not sure if that would have an affect.

@vroldanbet
Copy link
Contributor

vroldanbet commented Nov 7, 2023

Thanks for the repro! I created a new Rails application, imported the latest authzed-rb gem, set some breakpoints here and there, and identified the actual underlying error:

#<Google::Protobuf::TypeError: Unable to build file to DescriptorPool: Depends on file 'validate/validate.proto', but it has not been loaded>

This is going to require some research, as validate is now deprecated in favor of buf's fork.

EDIT: I couldn't find a quick way to address this. proto-validate does not offer ruby templates for protoc so until now the generated code has ignored the dependecy, thanks to an empty module to satisfy the import. The new Ruby code expects proto validate types to exist and complains they do not:

add_serialized_file verifies that all dependencies listed in the descriptor were previously added with add_serialized_file. Generally that should be fine, because the generated code will contain Ruby require statements for all dependencies, and the descriptor will fail to load anyway if the types depended on were not previously defined in the DescriptorPool.

The solution I proposed was to vendor the validate.proto file into the project and generate code out of it. See #79

vroldanbet added a commit that referenced this issue Nov 7, 2023
Ruby protobuf generator changes described in https://protobuf.dev/news/2023-04-20/
means the generated code is a bit more strict about dependencies and type definitions.

This fixes an error reported in runtime, for example, when using in a Rails application

```
Warning: Protobuf detected an import path issue while loading generated
file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/core_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
```

Inspecting the TypeError exception with a debugger revealed:

#<Google::Protobuf::TypeError: Unable to build file to DescriptorPool:
Depends on file 'validate/validate.proto', but it has not been loaded>

`protoc-gen-validate` does not offer ruby templates for protoc, so for `authzed-rb` the workaround has been to
generate an empty stub to satisfy the import, but that does not work anymore
with the new generated Ruby code.

This commit uses buf to generate the Ruby code for validate/validate.proto and satisfy the
missing dependency in the generated authzed API code
vroldanbet added a commit that referenced this issue Nov 7, 2023
Ruby protobuf generator changes described in https://protobuf.dev/news/2023-04-20/
means the generated code is a bit more strict about dependencies and type definitions.

This fixes an error reported in runtime, for example, when using in a Rails application

```
Warning: Protobuf detected an import path issue while loading generated
file /usr/local/rvm/gems/ruby-3.1.4/gems/authzed-0.7.0/lib/authzed/api/v1/core_pb.rb
Each proto file must use a consistent fully-qualified name.
This will become an error in the next major version.
```

Inspecting the TypeError exception with a debugger revealed:

#<Google::Protobuf::TypeError: Unable to build file to DescriptorPool:
Depends on file 'validate/validate.proto', but it has not been loaded>

`protoc-gen-validate` does not offer ruby templates for protoc, so for `authzed-rb` the workaround has been to
generate an empty stub to satisfy the import, but that does not work anymore
with the new generated Ruby code.

This commit uses buf to generate the Ruby code for validate/validate.proto and satisfy the
missing dependency in the generated authzed API code
@vroldanbet
Copy link
Contributor

@chiperific this should be fixed in https://github.com/authzed/authzed-rb/releases/tag/v0.7.1

@chiperific
Copy link
Author

chiperific commented Nov 8, 2023

Can confirm that 0.7.1 has no protobuf errors. Thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants