Extras not added as dependencies for mixed-case packages

[mattoberle]

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

Not to my knowledge.

Description

• PyPI distributions are case-insensitive and treat hyphens and underscores as equivalent (per PEP 426).
• pip-compile, the recommended tool for building requirements_lock.txt, always converts package names to lowercase.
• The whl.name value in bazel.extract_wheel derives from metadata in the package.

Given the above, dependencies are not linked in certain scenarios.

whl.name  # SQLAlchemy
extras  # {"sqlalchemy": {'postgresql_psycopg2binary'}}

extras_requested = extras[whl.name] if whl.name in extras else set()
whl_deps = sorted(whl.dependencies(extras_requested))

extras_requested  # set()

We need to apply some degree of sanitization to the keys of the extras dict and the lookup value to avoid the problem.
Or have extras be a dict class with a custom __getitem__ and __contains__ (not sure what would have the smaller impact).
I've applied a simple .lower() to whl.name as a patch in our WORKSPACE.
That works for us because our requirements_lock.txt is all lowercase, but that isn't a universal solution.

🔬 Minimal Reproduction

• https://github.com/mattoberle/rules_python_example

Running bazel test :test will fail with:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //:test
-----------------------------------------------------------------------------
F.
======================================================================
FAIL: test_import_psycopg2 (__main__.ExampleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "bazel-out/k8-fastbuild/bin/test.runfiles/example/test.py", line 13, in test_import_psycopg2
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bazel-out/k8-fastbuild/bin/test.runfiles/example/test.py", line 15, in test_import_psycopg2
    self.fail("Package 'psycopg2' not found.")
AssertionError: Package 'psycopg2' not found.

----------------------------------------------------------------------
Ran 2 tests in 0.412s

FAILED (failures=1)

Running bazel query 'kind(py_library, deps(@pip_sqlalchemy//:pkg)) will output:

@pip_greenlet//:pkg
@pip_sqlalchemy//:pkg

If you switch to the patched branch the test will pass and the query will output:

@pip_greenlet//:pkg
@pip_psycopg2_binary//:pkg
@pip_sqlalchemy//:pkg

🌍 Your Environment

Operating System:

  
Ubuntu 20.04.4 LTS
  

Output of bazel version:

  
5.1.1
  

Rules_python version:

  
0.8.1
  

Anything else relevant?

I'm happy to open a corresponding PR for this issue.

[thundergolfer]

Closing this as fixed in #724