Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: isolate symbols of Python extensions on Linux #3900

Closed

Conversation

alalek
Copy link

@alalek alalek commented Nov 16, 2017

to avoid conflicts with other loaded libprotobuf.so libraries.

resolves #3899

Python bindings extension and other versions of protobuf
(like system libbprotobuf.so)

No need to make protobuf symbols dynamic in Python extension.
Lets keep them local.
@grpc-kokoro
Copy link

Thanks for your pull request. The automated tests will run as soon as one of the admins verifies this change is ok for us to run on our infrastructure.

1 similar comment
@grpc-kokoro
Copy link

Thanks for your pull request. The automated tests will run as soon as one of the admins verifies this change is ok for us to run on our infrastructure.

@haberman
Copy link
Member

The risk of this approach is: if someone gets a google::protobuf::Message* from Python somehow, then tries to pass it to another protobuf API, we could get a crash then because the two protobuf library versions could have different internal layouts for objects.

Mixing libprotobuf.so versions in the same process seems inherently dangerous. If we go down this path, we should ensure as strictly as possible that you can't get a google::protobuf::Message*, proto2::protobuf::Descriptor*, etc. from the Python library.

@alalek
Copy link
Author

alalek commented Nov 17, 2017

Thank you for taking a look on this!

Protobuf is very popular library (even more than we want really), so avoiding mixed usage of different libprotobuf versions is very hard sometimes:

  • for example, using proto3 formats with system libprotobuf.so 2.x
  • or Python extension case: it is build on another platform (manylinux) without any version of libprotobuf.so and after that this extension is reused on other multiple Linux systems (with or without own libprotobuf.so).

Main assumption here is that protobuf implementations are isolated via API interfaces (according this: #435 (comment)) and don't share libprotobuf entries at all.
Second assumption is that non-system protobuf library is linked statically (case of Python extension from Pypi).

I agree, that some detection of this on compile / link time should be better, but this is out of scope of this patch.

This patch is attempt to isolate/hide symbols of statically linked protobuf from Linux loader (to avoid linking them with external .so with similar symbols of libprotobuf including different version).

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Apr 9, 2018

Closing old issues.

@xfxyjwf xfxyjwf closed this Apr 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build[python]: avoid symbols conflicts with other loaded libprotobuf.so (Linux)
5 participants