Downstream packager unable to indicate "no isolation" for offline builds

[jaraco]

In jaraco/jaraco.packaging#7 and jaraco/jaraco.packaging#9 and jaraco/jaraco.packaging#10, downstream packagers are struggling with the isolated builds as created by build.utils.project_wheel_metadata. These packagers would like to perform offline builds (preventing access to the network for resolving dependencies). The project_wheel_metadata call does allow for a isolated=False, but in the case where it's being enacted programmatically, the downstream user doesn't have any control.

I could imagine that each consumer of project_wheel_metadata could expose an environment variable or other configuration mechanism to alter the behavior, but probably build should do that instead. If, for example, the definition was project_wheel_metadata(isolated=get_bool_env('BUILD_ISOLATED', True)), then the downstream consumers could set BUILD_ISOLATED=false to disable isolated builds (and put the onus on them to supply the dependencies).

[FFY00]

I think it can make sense, however, I think this should be something that the consumers have to explicitly opt into.

[nanonyme]

Seems like there's semantical clash here. Isolation (how pypa/build currently refers to it) is isolation from system packages. But this is isolation from download servers which is a different thing. (maybe we should use term offline rather than isolated?) So there probably should be separate switches.

[nanonyme]

One way to handle this use case is simply cut network access completely from build sandbox so network requests fail but understandibly not everyone's build system supports that.

[layday]

build doesn't support isolated offline builds at all. Offline implies no isolation.

[jaraco]

See jaraco/jaraco.packaging#11 where an affected user contributed to jaraco.packaging to elicit an environment variable to override the behavior and avoid attempting to do a build while offline. Unfortunately, that patch was at the wrong layer, and it leaves other projects like jaraco/pytest-checkdocs#19 still failing in offline builds.

I'd really like for build to offer these users a way to provide pre-built wheels for load_metadata so to avoid building it.

If build doesn't want to offer this functionality, maybe I'll write a pytest plugin that monkeypatches build.load_metadata for all offline users.

[jaraco]

One possible recommendation could be for these downstream consumers to keep build isolation and network isolation but to provide a server or directory of the dependencies (for an isolated build to succeed without network access).

Perhaps that functionality already exists with PIP_FIND_LINKS and PIP_NO_INDEX. Indeed, it does:

 @ env PIP_NO_INDEX=1 PIP_FIND_LINKS=. pip-run -v tempora -- -c pass
Looking in links: .
Processing ./tempora-5.5.1-py3-none-any.whl
Processing ./pytz-2024.1-py2.py3-none-any.whl (from tempora)
Processing ./jaraco.functools-4.0.0-py3-none-any.whl (from tempora)
Processing ./more_itertools-10.2.0-py3-none-any.whl (from jaraco.functools>=1.20->tempora)
Installing collected packages: pytz, more-itertools, jaraco.functools, tempora
Successfully installed jaraco.functools-4.0.0 more-itertools-10.2.0 pytz-2024.1 tempora-5.5.1

(without PIP_FIND_LINKS, that command fails to find anything)

It's not obvious to me whether the use case is that they have wheels lying around for the dependencies or if they merely have the dependencies installed in some environment. If the dependencies are already installed somewhere and there are no wheels available, or if there's no installer (e.g. pip) available, then the build will still not succeed.

I think it can make sense, however, I think this should be something that the consumers have to explicitly opt into.

That's right. The proposed behavior above is no change to the status quo, but allows offline builders to set BUILD_ISOLATED=False to opt out of the default isolation.

One way to handle this use case is simply cut network access completely from build sandbox so network requests fail but understandibly not everyone's build system supports that.

Some builders already have the ability to cut network access completely, but that still doesn't help when build attempts to create an isolated environment and then doesn't have access to the network to resolve the dependencies declared by the build. If on the other hand, they've preemptively installed the dependencies into system site packages (or some other environment), which they've done, all they need to do is disable isolation and everything works.

build doesn't support isolated offline builds at all. Offline implies no isolation.

The corelary is that "no isolation" enables offline. Hence I want to give users the ability to disable isolation without requiring every library that uses project_wheel_metadata to solicit its own environment variable and plumb it through to build.

PR incoming.

[jaraco]

If build doesn't want to offer this functionality, maybe I'll write a pytest plugin that monkeypatches build.load_metadata for all offline users.

I realized a pytest plugin won't work, because jaraco.packaging is a sphinx plugin and pytest isn't relevant. It'd need to be its own package/library. Perhaps jaraco.packaging could actually provide this wrapper.

[jaraco]

This issue is addressed by jaraco.packaging >=9.5 in jaraco.packaging.metadata.load. Set BUILD_ENVIRONMENT=current and then invoke the tests/docs and the metadata will bypass the isolated build when loading metadata.