Python Wheel using dynamic linking (e.g. built with --cpp_implementation not --cpp_implementation --compile_static_extension)

[rowillia]

What language does this apply to?
Python
If it's a proto syntax change, is it for proto2 or proto3? N/A
If it's about generated code change, what programming language? Python

Describe the problem you are trying to solve.

Trying to use protobuf with cpp_implementation in the same python process as a library that's also using protobuf under the covers.

This seems to be a widespread issue in the tensorflow community. My issue isn't with tensorflow but the problem is identical. Tensorflow seems to have fixed this by suggesting people pin to an older version of protobuf before the wheels started shipping the native extension.

(e.g. tensorflow/tensorflow#8394,
tensorflow/tensorflow#9525 (comment)
tensorflow/tensorflow#24976,
tensorflow/tensorflow#35573,
https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/contrib/makefile/rename_protobuf.sh,
tensorflow/tensorflow#16104)

Describe the solution you'd like
An option to install a wheel which relies on dynamic linking instead of static linking, allowing both libraries to reference protobuf without causing segfaults and undefined behavior.

I think this could take a few forms:

• We could add the native extension as a an extra (e.g. users would pip install protobuf[cpp] to get the cpp extension).
• We could ship libprotobuf.*.{dylib, so, dll} as package_data and explicitly load it at import time with cdll which would allow other libraries to also find and load them.
• We could publish another package all together (e.g. protobufdynamic)

Describe alternatives you've considered

• Tried loading protobuf symbols from _message.*.so instead of libprotobuf`- not all symbols were baked in and seems overall risky/unstable
• Tried a similar approach to tensorflow and renaming protobuf itself (e.g. https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/contrib/makefile/rename_protobuf.sh)

Additional context
OSX defaults to RTLD_GLOBAL when loading libraries -https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/dlopen.3.html

[rowillia]

After further research #8346 should obviate the need for this.