-
Notifications
You must be signed in to change notification settings - Fork 15.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python] A file with this name is already in the pool. #3002
Comments
I ran into this issue too. I have to make the |
Ran in to this issue. If anyone finds this, I resolved it by running |
I found that doing
before running my Python code resolved the issue. I got the flag from the bottom of https://developers.google.com/protocol-buffers/docs/reference/python-generated |
The error (file already in pool) indicates an actual problem you need to fix on the .proto files. A simple solution is to put one of the .proto file in a subdirectory. Right now you can hide the problem by switching to the pure python implementation (which doesn't check conflicts), but that won't really solve the issue. You are going to run into the same problem if you are using a different language, or if we fix the pure python implementation to check conflict as well. |
@xfxyjwf I think you are confused. The original issue cannot be fixed by doing anything the protos, except renaming the duplicate message. The pure python implementation uses the directory for namespacing, and that works. Just as described in the documentation https://developers.google.com/protocol-buffers/docs/proto#packages The problem is that the C++ implementation doesn't do that, nor does it respect the |
@rossengeorgiev Can you regenerate the *_pb2.py files by invoking: $ protoc --python_out=. module1/messages.proto module2/messages.proto ? That should solve the issue you are having. Note that the python import path must match the file path you passed to protoc. |
I managed to figure out what caused the original issue. The clue was in the error. The descriptor name is derived from the relative path.
That means I compiled the code by going into each directory and running
In addition if we have duplicate messages like so: ./module1/messages.proto syntax = "proto2";
message mymessage { } ./module2/messages.proto syntax = "proto2";
message mymessage { } If we invoke protoc like so, we get an error and nothing gets compiled. protoc --python_out=. module2/messages.proto module1/messages.proto
module1/messages.proto:3:9: "mymessage" is already defined in file "module2/messages.proto". Alternatively, we could do: protoc --python_out=. module2/messages.proto
protoc --python_out=. module1/messages.proto No error and compiles fine. However we get an error when we try to import in the C++ protobuf package, and no error in the pure one. $ python -c "import module1.messages_pb2, module2.messages_pb2"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "module2/messages_pb2.py", line 22, in <module>
serialized_pb=_b('\n\x16module2/messages.proto\"\x0b\n\tmymessage')
File "/home/vagrant/env/local/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 824, in __new__
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "module2/messages.proto":
mymessage: "mymessage" is already defined in file "module1/messages.proto". At this point we have two options. Use pure python protobuf package, or set |
@rossengeorgiev could you clarify what you mean by "set package directive in our |
I was working with tensorflow protos that I was trying to compile and use outside of tensorflow- there were a couple of issues but the main issue seems to have been not making sure I had uninstalled tensorflow and from my python environment entirely because one of my services was using a tf service which was causing the conflict- after I refactored that to not use tf and got rid of tf for good measure, I stopped seeing this issue |
to workaround this kind of problem when using the module on projects that also import `fonts_public_pb2.py`: protocolbuffers/protobuf#3002
to workaround this kind of problem when using the module on projects that also import `fonts_public_pb2.py`: protocolbuffers/protobuf#3002
to workaround this kind of problem when using the module on projects that also import `fonts_public_pb2.py`: protocolbuffers/protobuf#3002
Hi, I was facing this issue on adding a dependency and tried using
|
Years later and it's still unclear from this conversation what the proper solution is. Avoiding use of the binary wheels by forcing pip to install the pure python implementation is a workaround. What is the proper fix? @rossengeorgiev said:
Setting the ./foo/messages.proto syntax = "proto2";
package foo; ./bar/messages.proto syntax = "proto2";
package bar; And I still get this error:
|
Agreed @dkbarn, still having the same issue, this is definitely not solved. The pure python implementation workaround it is obviously much slower at runtime. @jtattermusch @BSBandme |
This problem arises from the linux binary wheels for
protobuf==3.2.0
. Using the pure python implementation works fine.In my case I proto file that collides with name of proto file from another package. Importing both python packages is not possible as I get this error.
Steps to replicate
Directory tree:
./module1/messages.proto
./module2/messages.proto
Trying to import:
The text was updated successfully, but these errors were encountered: