[BUG] ImportError on import buf/validate

[o-murphy]

Description

if we generate the python sources with buf and --include-imports flag, the protovalidate tries to import buf locally, but if we want to make generated sources a part of other module, the protovalidate raises ImportError on buf/validate import, so we need modify the protovalidate module sources to set where we want to import buf/validate

[rodaine]

Hey @o-murphy! Thanks for the report. Can you give more details about the steps you're running and the errors you are seeing?

[convoi]

I believe I have the same problem.
My source code is structured like

src/my_module/stuff.py
protos/events/v1/some.proto

my buf.gen.yaml:

# buf.gen.yaml
version: v1

plugins:
  - plugin: buf.build/protocolbuffers/python:v21.9
    out: ./src/my_module
  - plugin: buf.build/protocolbuffers/pyi:v21.9
    out: ./src/my_module

I generate my python proto files into my module:

buf generate --include-imports protos

This puts the generated python classes and the buf validate files in

src/my_module/events
src/my_module/buf/validate

so if I wanted to importe the validate_pb2.py I need to do

 from my_module.buf.validate import expression_pb2

but protovalidate wants from buf.validate import expression_pb2.

(Disclaimer, to make generated python proto classes work with this module src layout you have to do a sed rewrite trick - I was just hoping that I do not need some other hacks for validation. Also here it would be trickier since I would have to rewrite protovalidate module )

[rodaine]

Got it! Yes this is a known issue (also reported on buf) for protobuf python generation in general. There are some workarounds in that linked thread, but without a rewrite or a sys.path.insert, you'll need to generate into the root of the python project.

[o-murphy]

@rodaine

I want to distribute my project to PyPi, I use protovalidate as a dependency. I also want to generate *_fb2.py and buf files and distribute it as a part of my project, so I need to move generated *_fb2.py files and buf directory to the top-level package's dir. But in this case I get an import error if I use import protovalidate in one of the inner modules, cause the protovalidate tries to import the buf from the project root dir but not from the package top-level dir. It happen even if I change the import paths in a *_pb2.py files.

Now I have a temporary sollution: copying the protovalidate directly to my library and fix the import's in each module of protovalidate to solve it, for example:

# protovalidate.validator.py

from a7p.buf.validate import expression_pb2  # type: ignore
from a7p.protovalidate.internal import constraints as _constraints
from a7p.protovalidate.internal import extra_func

Can you provide me a way how can I set the top-level of my library to protovalidate module to get the valid import of buf module?

Look at this project tree please
In my case: pya7p - the project root and the a7p - the package top level dir