Stack overflow when reflecting mutually-referential messages

[zhihuizhang17]

Summary
When a proto file defines multiple messages that reference each other (e.g., request/response pairs), asking the reflection server to describe one of those messages
eventually crashes with a stack overflow. The reflection builder currently emits one pseudo proto file per message. For two messages in the same source file that depend on each other, the client chases an endless loop of FileByFilename lookups between the synthetic files and never settles.

Steps to Reproduce

1. Edit examples/helloworld_v3/priv/protos/helloworld.proto and ensure HelloRequest references another message in that file—for example add google.protobuf.Struct
   struct = 2;. (This mirrors the scenario where two messages in one proto depend on each other or on a shared nested type.)
2. Regenerate the Elixir stubs so the updated proto is reflected in the .pb.ex files.
3. Start the reflection server then call it (use grpcurl: grpcurl -plaintext localhost:50051 describe .helloworld.HelloRequest).
4. Observe the client output with runtime: stack overflow. The captured stack trace is in result.txt.

result.txt

Expected Behaviour
Asking for file_containing_symbol (or file_by_filename) should return the real proto (helloworld.proto) once, listing both HelloRequest and HelloReply in message_type with a finite dependency list.

[mjheilmann]

should return the real proto (helloworld.proto) once

this demonstrates that using the elixir module source file may not be reliable, without additional instruction/requirements on the users to not use that option while still including the descriptors.

If we are providing more (or changing) user directives, perhaps we should consider other alternatives as well

[mjheilmann]

Assuming that gRPC proto files might not be 1-1 with elixir files, and that we might not be able to detect this change from our users, I can think of some alternatives that might be worth looking into:

first option
using the symbol path from the descriptor, combine similar namespaces into single files. i.e. helloworld.Greeter, .helloworld.HelloRequest, and .helloworld.HelloReply are combined into the namespace helloworld, while .google.protobuf.Timestamp is recognized as a different namespace.

This will end up with bloated responses as each namespace is combined into a single large response and file.

second option
look at https://github.com/elixir-protobuf/protobuf to see if we can add additional information to compiled modules, where the proto files are known to be accessible, so we can include that information in the pb.ex modules.

I don't view this as very possible, this information isn't useful outside of our use case. This will also have the same problem the linked PR has, where we need to know/discover what parts of the path are proto namespace, and which parts are the users filesystem.

option three
Add a pre-compile step where grpc_refection receives the proto paths from the user as a compile-time option, and we scrape the files directly. This should remove the need for the descriptors, but:

• requires the protos to be available at compile-time, which is not a current requirement. It might be difficult for users who store and process their protos separately from their elixir code
• We could end up duplicating work from protobuf, where they already create descriptors from source

more options are welcome of corse. I think the first option seems the simplest, ignoring the bloat we will already know how to do this and it doesn't try t match the original proto files exactly

[PragTob]

👋 Sorry too many notifications. I see there's questions on the PR and options lined out, but I don't know the internals library well.

I'm making a mental note to check back next week from work, let me know if I can help.