hatchling: plugin handling issue when build isolation is off

[Hedwyn]

Hello,
I've been getting some issues with hatch failing in no-build-isolation mode when using hatch plugins (when no-build-isolation with pip).
To summarize the issue, here's the configuration for which it happens (python 3.12.9)

• Everything is run without a single virtual environment
• Package 1 uses hatchling backend with hatch-vcs plugin and another custom plugin CustomPlugin.
• Package 2 uses hatchling backend with only hatch-vcs plugin
• Package 1 gets installed with --no-build-isolation - works fine
• Then when trying to install Package 2 with no-build-isolation, the install fails on trying to load the version (and more specifically trying to load hatch-vcs). Failure happens because it fails to import the custom plugin... even though 1) Package 2 does not use it and has no relation with CustomPlugin and 2), the plugin is actually installed in the virtual environment and build isolation should be off.

Note: frontend used it here is just straight calls to pip, but issue is the same using hatch frontend.

So tracking the issue down the issue here seems to be the following: hatchling uses pluggy for the plugin architecture, and more specifically when the logic loading the plugins uses load_setuptools_entrypoints from pluggy: https://github.com/pytest-dev/pluggy/blob/main/src/pluggy/_manager.py#L391. This function just iterates over all the dist-info found and tries to find all the hatch entrypoints. When looking for plugins, hatchling loads all the one if finds installed and then tries to get them by name.
In my case, while loading every plugin available it finds the hatch entrypoint added by CustomPlugin - which is normal so far, since build isolation is off, the build for Package 1 is leaky here.
The problem is actually that pluggy then tries to load the plugin (https://github.com/pytest-dev/pluggy/blob/main/src/pluggy/_manager.py#L415) and the import fails (and again - package is actually installed in the virtual environment and the import works perfectly fine from there). It looks like at that stage of the build the environment in which hatch runs has a dist-info for a package that's not available (and also means that the environment is not the one from which it was called...).
As far as it happens within pluggy logic, I suspect the root issue is upstream, as it looks like there some isolation going on but this isolation is leaky (getting dist-info from packages that are not actually available in the environment used at that time ?). I'm not sure if that isolation is normal as the build is run with --no-build-isolation and it it is there might be an issue with the implementation. I suspect it could be related to some caching side effects (the isolated environement has caching artifacts containing this dist-info), but I'm not familiar enough with hatchling internal implementation to tell.

[Hedwyn]

Update: I found out that the context required for the issue is a bit more specific than what I described above.
I realized that hatch was attempting to load CustomPlugin not because it was installed in the venv but rather because there's a third project Package 3, installed in the same environement, that has a hatch entrypoint calling that CustomPlugin.
The behavior is still wrong though since:

• Package 3 has also nothing to do with Package 2 and is not involved in its installation (thus is creating side effects on completely unrelated projects)
• And we still get a leaky isolation where somehow the plugin loading logic uses dist-info from packages that are not actually available from the calling environment (here, Package 3 and CustomPlugin), I would guess due to some dist-info caching.