REF: Eliminate discovery hook fallback in Stage 2 build pipeline

[LalatenduMohanty]

During Stage 2 builds (build-sequence / build-parallel), if cached requirement files (build-system-requirements.txt, build-backend-requirements.txt, build-sdist-requirements.txt) are missing, fromager silently falls back to running PEP 517 discovery hooks and making network requests. This defeats the purpose of separating discovery from compilation.

The fallback happens in three functions in dependencies.py:

• get_build_system_dependencies()
• get_build_backend_dependencies()
• get_build_sdist_dependencies()

[tiran]

The hooks don't have access to network:

• default_get_build_system_dependencies does not run a PEP 517 hook. It just reads build-system requires field from pyproject.toml
• default_get_build_backend_dependencies and default_get_build_sdist_dependencies use get_build_backend_hook_caller. The backend caller runs in a subprocess with network isolation.

[tiran]

PS: I think you have found a design flaw. The build stage should not use the PEP 517 hooks to detect build requirements. Instead it should use the information from the build graph to populate the build environment.

[dhellmann]

PS: I think you have found a design flaw. The build stage should not use the PEP 517 hooks to detect build requirements. Instead it should use the information from the build graph to populate the build environment.

Yes, the graph needs to be the source of truth for build details. It currently includes a source URL, but I don't know if that's accurate for all resolver types. For example, a requirement with a git URL needs to ensure that it points properly to the git tag or SHA in the repo. We may need to add information to the nodes in the graph to support that, and that information might need to take different forms for different packages.

[LalatenduMohanty]

Here are some ways we can achieve this

1. Hardcode no-discovery in build commands

   build-sequence and build-parallel are Stage 2 commands — they should always expect cache files from a prior bootstrap. Just disable discovery unconditionally in those code paths. If cache files are missing during a build, it's always an error — there's no legitimate reason to fall back to PEP 517 hooks at that point.

2. Populate cache from the dependency graph before building

   The dependency graph already stores build deps via DependencyEdge.req_type (BUILD_SYSTEM, BUILD_BACKEND, BUILD_SDIST). The build commands could write the cache files from graph.json before building each package, so the cache-hit path always succeeds and discovery is never invoked. This aligns with the understanding that the graph should be the source of truth.

3. Pass allow_discovery at the call site (no global state)

   Instead of a flag on WorkContext, have prepare_build_environment() accept an allow_discovery parameter. Bootstrap passes True, build commands pass False. Keeps the concern local rather than on shared state.

4. CLI flag --no-discovery

   Add a user-facing flag to build commands. Works, but makes the user responsible for preventing a behavior that should arguably never happen in Stage 2.

[tiran]

• fromager build is a debug and test command. It does not have access to a graph file and therefore needs to use the current implementation of prepare_build_environment.
• fromager build-sequence and fromager build-parallel have access to a graph file. They should use DependencyNode.iter_build_requirements to fill the build environment with build dependencies.

[LalatenduMohanty]

Thanks @tiran here is the proposal for using the graph to populate the build environment directly.

Current flow (build-sequence / build-parallel):

_build() → prepare_build_environment() → get_build_{system,backend,sdist}_dependencies()
                                              ↓
                                    cache miss → run PEP 517 hooks

Proposed flow:

build_parallel / build_sequence
    ↓
    extract build deps from graph via DependencyNode.iter_build_requirements()
    ↓
    pass pre-resolved deps to _build()
    ↓
    _build() → new build env setup path that installs given deps directly

Does this direction look right?