Option to exclude site-packages from python path

[jmfederico]

PEP 582, as it is today only adds __pypackages__ to the list of discovery paths. This might lead to the usage of globally installed packages by mistake.

I think it would be very useful if it were possible to remove global python paths on a per project basis. Maybe a flag on the pyproject.toml file.

Assuming a project in /Users/foo/projects/bar on macOS using python3.8 installed using official installers:

• Current path list

  ['',
   '/Users/foo/projects/bar/pdm/__pypackages__/3.8/lib',
   '/Users/foo/projects/bar/pdm',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
   '/Users/foo/Library/Python/3.8/lib/python/site-packages',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages'
  ]

• Suggested path list

  ['',
   '/Users/foo/projects/bar/pdm/__pypackages__/3.8/lib',
   '/Users/foo/projects/bar/pdm',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8',
   '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload',
  ]

Is this something that you are open to?

[frostming]

The behavior should affect two places:

1. pdm run
2. The entry point scripts under __pypacakges__/X.Y/bin

While we can provide an option in the first situation, we can't make this behavior as optional in the second situation.
So I would like to disable site-packages by default, any concern? @jmfederico

[jmfederico]

@frostming it works for me. I do think that having system wide packages available with locally installed packages is a recipe for disaster.

[frostming]

Still waiting for the CI to pass(something is going wrong with GitHub actions), the next release should come out in the next 2 days after the PR is merged.

[mgzenitech]

Just tested this and still get site-packages in sys.path. Is this really fixed to exclude them?

pyproject.toml

[tool.pdm]
    version = { use_scm = true }

    [[tool.pdm.source]]
        name = "pypi"
        url = "https://pypi.org/simple"
        verify_ssl = true

    [tool.pdm.scripts]
        _.env_file = ".env"
        cli = { call = "lib.python.__pdm.commands.cli:cli()" }

.env

PYTHONPATH=lib/python:__pypackages__/3.9/lib
PYTHONDONTWRITEBYTECODE=1

lib/python/__pdm/commands/cli.py

import sys

def cli() -> None:
    print(sys.path)

pdm run cli result:

['', '/home/mariusg/Documents/Projects/infra-terraform/lib/python', '/home/mariusg/Documents/Projects/infra-terraform/__pypackages__/3.9/lib', '/home/mariusg/.pyenv/versions/3.9.9/lib/python39.zip', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/lib-dynload', '/home/mariusg/.local/lib/python3.9/site-packages', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/site-packages']

[frostming]

@mgzenitech If you use pdm run, they are excluded by default, and we offer an option to include them, but if you use the global PEP 582, they are always included.

[mgzenitech]

Hi @frostming !
I haven't enabled PEP 582 globally (did not add anything to rc files from pdm --pep582). Also as you can see I am setting my own PYTHONPATH from .env file
PYTHONPATH=lib/python:__pypackages__/3.9/lib

[frostming]

@mgzenitech The PYTHONPATH looks super weird to me.

I never have to set it like that. for example __pypackages__/3.9/lib will be auto-added when pdm run.

[mgzenitech]

@frostming __pypackages__/3.9/lib is added only if you are not using any additional paths (in my case I need to add lib/python via env variable)

[frostming]

@mgzenitech > is added only if you are not using any additional paths

That is not correct, why do you think so?

[mgzenitech]

@frostming because I have tried it and it didn't add __pypackages__/3.9/lib. Initially I've tried this in my .env file:

PYTHONPATH=lib/python
PYTHONDONTWRITEBYTECODE=1

And no packages from __pypackages__/3.9/lib could be imported

result from sys.path:

['', '/home/mariusg/Documents/Projects/infra-terraform/lib/python', '/home/mariusg/.pyenv/versions/3.9.9/lib/python39.zip', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/lib-dynload', '/home/mariusg/.local/lib/python3.9/site-packages', '/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/site-packages']

[frostming]

i might know why, you are using a venv Python. check it by 'pdm info', the packages path shouldnt be empty @mgzenitech

[mgzenitech]

@frostming

➜  infra-terraform git:(pdm) ✗ pdm info
PDM version:        1.15.4
Python Interpreter: /home/mariusg/.pyenv/versions/3.9.9/bin/python3.9 (3.9)
Project Root:       /home/mariusg/Documents/Projects/infra-terraform
Project Packages:   /home/mariusg/Documents/Projects/infra-terraform/__pypackages__/3.9

[frostming]

@mgzenitech it turns out to be a corner case that the running env overrides the preset ones. It is handled in #1211

[mgzenitech]

@frostming superb!