Skip to content

Commit

Permalink
Attempting to have a different proto_path and output_path
Browse files Browse the repository at this point in the history
This is me attempting to follow the advice on
CZ-NIC/setuptools-grpc#16 (comment),
but this doesn't really work.

First build doesn't work (note that there are no `module1_pb2.*` files):

    $ pip install . && pip show -f jfly
    Processing /home/jeremy/tmp/setuptools-grpc-demo
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
      Installing backend dependencies ... done
      Preparing metadata (pyproject.toml) ... done
    Building wheels for collected packages: jfly
      Building wheel for jfly (pyproject.toml) ... done
      Created wheel for jfly: filename=jfly-0.0.1-py3-none-any.whl size=904 sha256=c7eed92a4897aca10015c53a8a81b552a36bb2b5629561a37643829907516819
      Stored in directory: /home/jeremy/.cache/pip/wheels/94/3f/96/34cbbd64f469cfadb00e57c6f6651a098fe76c85f79dbb0e9c
    Successfully built jfly
    Installing collected packages: jfly
    Successfully installed jfly-0.0.1

    [notice] A new release of pip is available: 23.2.1 -> 23.3.2
    [notice] To update, run: pip install --upgrade pip
    Name: jfly
    Version: 0.0.1
    Summary:
    Home-page:
    Author:
    Author-email:
    License:
    Location: /home/jeremy/tmp/setuptools-grpc-demo/.direnv/python-3.11/lib/python3.11/site-packages
    Requires:
    Required-by:
    Files:
      jfly-0.0.1.dist-info/INSTALLER
      jfly-0.0.1.dist-info/METADATA
      jfly-0.0.1.dist-info/RECORD
      jfly-0.0.1.dist-info/REQUESTED
      jfly-0.0.1.dist-info/WHEEL
      jfly-0.0.1.dist-info/direct_url.json
      jfly-0.0.1.dist-info/top_level.txt

Second build *does* work (note that there are `module1_pb2.*` files):

    $ pip install . && pip show -f jfly
    Processing /home/jeremy/tmp/setuptools-grpc-demo
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
      Installing backend dependencies ... done
      Preparing metadata (pyproject.toml) ... done
    Building wheels for collected packages: jfly
      Building wheel for jfly (pyproject.toml) ... done
      Created wheel for jfly: filename=jfly-0.0.1-py3-none-any.whl size=2905 sha256=45e70cd416572af4c4df2fa9d4b293f2df863ae46525b868a97ffd9900b75eba
      Stored in directory: /home/jeremy/.cache/pip/wheels/94/3f/96/34cbbd64f469cfadb00e57c6f6651a098fe76c85f79dbb0e9c
    Successfully built jfly
    Installing collected packages: jfly
      Attempting uninstall: jfly
        Found existing installation: jfly 0.0.1
        Uninstalling jfly-0.0.1:
          Successfully uninstalled jfly-0.0.1
    Successfully installed jfly-0.0.1

    [notice] A new release of pip is available: 23.2.1 -> 23.3.2
    [notice] To update, run: pip install --upgrade pip
    Name: jfly
    Version: 0.0.1
    Summary:
    Home-page:
    Author:
    Author-email:
    License:
    Location: /home/jeremy/tmp/setuptools-grpc-demo/.direnv/python-3.11/lib/python3.11/site-packages
    Requires:
    Required-by:
    Files:
      jfly-0.0.1.dist-info/INSTALLER
      jfly-0.0.1.dist-info/METADATA
      jfly-0.0.1.dist-info/RECORD
      jfly-0.0.1.dist-info/REQUESTED
      jfly-0.0.1.dist-info/WHEEL
      jfly-0.0.1.dist-info/direct_url.json
      jfly-0.0.1.dist-info/top_level.txt
      out/foo/__pycache__/module1_pb2.cpython-311.pyc
      out/foo/__pycache__/module1_pb2_grpc.cpython-311.pyc
      out/foo/module1_pb2.py
      out/foo/module1_pb2.pyi
      out/foo/module1_pb2_grpc.py

I haven't read all the relevant setuptools code yet, but I believe
setuptools does its package autodiscovery (possibly during the `egg_info` command?)
*before* invoking the `build` command, so on the first run, it doesn't
find any packages, and on the second run (with the `output_path` already
populated), it does discover the files.

I briefly played with a couple of ideas:

1. Explicitly listing the packages.
  - This doesn't work: the `egg_info` command crashes complaining that
    these package directories don't exist in the `out` directory.
2. Trying to hook earlier into the build process (before the `egg_info`
   command).
   - This didn't immediately work. I don't know enough about the details
     of python packaging to know if this idea has legs.
  • Loading branch information
jfly committed Jan 12, 2024
1 parent 1af5d21 commit 6508482
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/.direnv
/build
/src/*.egg-info
/*.egg-info
4 changes: 4 additions & 0 deletions out/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this file
!.gitignore
5 changes: 4 additions & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,7 @@
proto_files = **/*.proto
grpc_files = **/*.proto
proto_path = ./src
output_path = ./src
output_path = ./out

[options]
packages = find_namespace:

0 comments on commit 6508482

Please sign in to comment.