Allow filenames in _CalledFromGeneratedFile to contain "/pb2/"

[twmr]

What language does this apply to?
python (c++ implementation)

Describe the problem you are trying to solve.

The company I work with strips the _pb2 suffix from the filenames of the generated python modules. Instead a pb2 python subpackage (e.g. companyname.pb2) is created, which denotes that the modules in this pb2 subpackage are generated by protoc (and postprocessed).

If the c++ implementation of the protobuf bindings is used, importing those generated modules does not work:

    652 
    653     def __new__(cls, name, index, number, type=None, options=None):
--> 654       _message.Message._CheckCalledFromGeneratedFile()
    655       # There is no way we can build a complete EnumValueDescriptor with the
    656       # given parameters (the name of the Enum is not known, for example).

TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Describe the solution you'd like
Given that a similar bug/feature request was recently fixed (#4262), I would like that _CalledFromGeneratedFile returns true when the filename contains the string "/pb2/".

Describe alternatives you've considered

Additional context
Add any other context or screenshots about the feature request here.

[anandolee]

Do you know why your company strips the _pb2 suffix?

#4262 is for cython which is a common issue. I don't think it make sense to make a fix if it is only an issue that caused by your company's change.

[twmr]

Yes, I know. It was me who created this companyname.pb2.xxx package hierarchy format for our compiled proto-files years ago.

I created this convention because I think that this hierarchy is more pythonic than having modules suffixed with _pb2.

[twmr]

There is a comment in https://github.com/google/protobuf/blob/HEAD/python/google/protobuf/pyext/descriptor.cc#L94

suggesting to completely remove the (hacky) _CalledFromGeneratedFile function. Are there any updates on this?

[twmr]

@anandolee what do you think about this feature request? Does it make sense if I come up with a PR?

[anandolee]

I agree to remove this function, it is a behavior diffs between pure python and c extension as well.

Add @amauryfa on the TODO-removal, do we still have something to be worried for the removal?

[twmr]

FYI: I've started to work on the removal of _CalledFromGeneratedFile in twmr@e0ae302 (there are some python tests that fail with this change: https://travis-ci.org/thisch/protobuf/jobs/428294435)

[anandolee]

Those tests are for test coverage and also mark the behavior diffs of pure python and cpp extension. Could remove the failure tests or change to new behavior.

[anandolee]

Close it as PR #5171 is closed. Feel free to reopen it is still needed