Installing a project with a path from the parent of PROJECT_ROOT fails with KeyError.

[rohit507]

I've got a series of projects in the same parent directory that seem to cause the same problem. Specifically when I'm trying to move up a directory from PROJECT_ROOT while keeping the pyproject.toml file location independent.

When I use pdm add -e ./../local-dependency the dependency is resolved to an absolute path that won't work when coworkers pull my changes.

Instead I've tried pdm add -e "file:///\${PROJECT_ROOT}/../local-dependency" which proceeds to fail exactly like the case without the .. in this issue.

• I have searched the issue tracker and believe that this is not a duplicate.

It's not clear to me that this is separate issue from #721, rather than an edge case that should be handled there. I made a comment in that thread but elected to create a new issue anyway because there's enough activity on this repo that a comment would likely get lost.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

Directory:

.
├── local-dependency
│   ├── __init__.py
│   └── pyproject.toml
└── local-dependency-2
    ├── __init__.py
    ├── pdm.lock
    └── pyproject.toml

local-dependency/pyproject.toml:

[project]
name = "my_lib"
version = "0.1"
description = ""
authors = [
    {name = "Me", email = "me@foo.bar"},
]
dependencies = []
requires-python = ">=3.9"
dynamic = ["classifiers"]
license = {text = "MIT"}

[project.urls]
homepage = ""

[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"

local-dependency-2/pyproject.toml:

[project]
name = "my_lib_2"
version = "0.1"
description = ""
authors = [
    {name = "Me", email = "me@foo.bar"},
]
dependencies = [
   "-e file:///${PROJECT_ROOT}/../local-dependency"
]
requires-python = ">=3.9"
dynamic = ["classifiers"]
license = {text = "MIT"}

[project.urls]
homepage = ""

[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"

local-dependency-2/pdm.lock:

[[package]]
name = "my-lib"
version = "0.1"
requires_python = ">=3.9"
editable = true
path = "./../local-dependency"
summary = "UNKNOWN"

[metadata]
lock_version = "3.1"
content_hash = "sha256:846ee7de62586775a93ebb56bb357769d9c7f0e955e391656c274c4898200315"

[metadata.files]

Actual behavior

pdm install, pdm add, similar operations fail with the following error:

> pdm install -v
~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Preparing isolated env for PEP 517 build...
Collecting pdm-pep517
  Using cached pdm_pep517-0.12.3-py3-none-any.whl (302 kB)
Installing collected packages: pdm-pep517
Successfully installed pdm-pep517-0.12.3
/tmp/pdm-build-env-hvtwo5xe-shared/lib/python3.9/site-packages/pdm/pep517/base.py:394: PDMWarning: `classifiers` no longer supports dynamic filling, please remove it from `dynamic` fields and manually supply all the classifiers
  for classifier in meta.classifiers or []:
Traceback (most recent call last):
  File "~/mambaforge/envs/simple-uam/bin/pdm", line 10, in <module>
    sys.exit(main())
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/core.py", line 233, in main
    return Core().main(args)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/core.py", line 168, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/core.py", line 163, in main
    f(options.project, options)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/cli/commands/install.py", line 49, in handle
    actions.do_sync(
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/cli/actions.py", line 193, in do_sync
    candidates = resolve_candidates_from_lockfile(project, requirements)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/cli/actions.py", line 140, in resolve_candidates_from_lockfile
    mapping, *_ = resolve(
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/resolver/core.py", line 31, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/resolver/providers.py", line 172, in get_dependencies
    deps, requires_python, _ = self.repository.get_dependencies(candidate)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/models/repositories.py", line 407, in get_dependencies
    reqs, python, summary = super().get_dependencies(candidate)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/models/repositories.py", line 70, in get_dependencies
    requirements, requires_python, summary = getter(candidate)
  File "~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/pdm/models/repositories.py", line 399, in _get_dependencies_from_lockfile
    return self.candidate_info[self._identify_candidate(candidate)]
KeyError: ('my-lib', '0.1', 'file:///${PROJECT_ROOT}/../local-dependency', True)

Expected behavior

The command should succeed and create a new environment as usual.

Environment Information

Version:

> pdm --version                                                
Python Development Master (PDM), version 1.14.1

Env:

> pdm info; and pdm info --env
~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
PDM version:        1.14.1
Python Interpreter: ~/mambaforge/envs/simple-uam/bin/python3 (3.9)
Project Root:       ~/Workspace/pdm-test/local-dependency-2
Project Packages:   ~/Workspace/pdm-test/local-dependency-2/__pypackages__/3.9
~/mambaforge/envs/simple-uam/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.12",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.4.0-107-generic",
  "platform_system": "Linux",
  "platform_version": "#121-Ubuntu SMP Thu Mar 24 16:04:27 UTC 2022",
  "python_full_version": "3.9.12",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

[frostming]

We just don't allow referencing a package beyond the project root with ${PROJECT_ROOT}. This will break the reproducibility of the project, which is one of the main goals of this package manager.

[gjoseph92]

@frostming I get the rationale for this, but it might be good to display a clearer user-facing error in this case. The KeyError is cryptic and doesn't make it clear that this isn't allowed (versus that you just formatted the string incorrectly).