Mypy suggests installing "types-requests" when it is already installed

[lucemia]

Bug Report

Got Library stubs not installed for "requests.packages.urllib3.util.ssl_" [import-untyped] error.
Mypy suggest install types-requests but types-requests is already installed

To Reproduce

store the following line as reproduce.py

from requests.packages.urllib3.util.ssl_ import create_urllib3_context

Execute

(genv-studio-py3.10) ➜  gstudio git:(dependabot/pip/mypy-1.6.1) ✗ mypy reproduce.py --install-types --non-interactive  
Installing missing stub packages:
/Users/davidchen/repo/gstudio/.venv/bin/python -m pip install types-requests

Requirement already satisfied: types-requests in ./.venv/lib/python3.10/site-packages (2.28.11.5)
Requirement already satisfied: types-urllib3<1.27 in ./.venv/lib/python3.10/site-packages (from types-requests) (1.26.25.4)

[notice] A new release of pip available: 22.3.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip

reproduce.py:1: error: Library stubs not installed for "requests.packages.urllib3.util.ssl_"  [import-untyped]
reproduce.py:1: note: Hint: "python3 -m pip install types-requests"
reproduce.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
reproduce.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

Expected Behavior

• no error (which is true in mypy 1.4.1)

Actual Behavior

• raise error: Library stubs not installed for "requests.packages.urllib3.util.ssl_" [import-untyped]

Your Environment

• Mypy version used: 1.6.1
• Mypy command-line flags: --install-types --non-interactive
• Mypy configuration options from mypy.ini (and other config files):

ignore_missing_imports = True
show_error_codes = True
implicit_optional = True
exclude = .*/(migrations|commands|scripts)/.*

• Python version used: 3.10.8

[JelleZijlstra]

The requests.packages directory was removed from requests itself and from our stubs, so I don't think the error is incorrect.

However, there's an actionable mypy bug here: the note suggests installing types-requests, when in fact it's already installed. We shouldn't do that.

[hauntsaninja]

Annoying to implement, you'd have to do a bunch of rooting around in dist-info and the like to figure out what the user actually has installed (e.g. this error could be caused by wrong version of types-requests or use of some other stubs package for requests).

I can leave the issue open, but this will probably be a won't fix in practice. And that's not the worst thing either, the relevant part of the message is just a hint

[jaraco]

I ran into this issue today as well for from requests.packages.urllib3.connectionpool import HTTPConnection in jaraco.financial.

you'd have to do a bunch of rooting around in dist-info and the like to figure out what the user actually has installed

I think a check could be as straightforward as:

import importlib.metadata as md

def is_installed(package):
    try:
        md.distribution(package)
    except md.PackageNotFoundError:
        return False
    return True

Or if you have jaraco.context:

is_installed = jaraco.context.ExceptionTrap(md.PackageNotFoundError).passes(md.distribution)

I was slightly surprised how difficult it is to code for "is a distribution present". That should probably be made easier in importlib metadata (python/importlib_metadata#477).

[jaraco]

On further consideration, and looking at the code, at the point where the hint is generated, the stub_dist isn't available, so it may require some refactoring to even adjust the hint.

But more importantly, I think there's a second concern, which is that mypy is detecting types-requests as "approved stubs" for requests.packages when types-requests doesn't have those stubs. I realize it's a can of worms to consider more than just top-level packages and their associated stubs, but it's not just the hint that's wrong but the reason. The reason should be NOT_FOUND and not APPROVED_STUBS_NOT_INSTALLED.

[jaraco]

The requests.packages directory was removed from requests itself and from our stubs, so I don't think the error is incorrect.

This doesn't seem to be the case for the latest stable requests:

 ~ @ pip-run requests -- -c "import requests.packages" && echo done
done

Searching around, I found things like psf/requests#4500, where the module/package appears to have been removed in 2018 slated for a 3.x release, but the current latest release is 2.31. So was it removed? I don't get any deprecation warnings or anything for my import statement, so I'm not sure what the replacement should be.