Install plugins from pyproject.toml (+config)

[tgolsson]

Is your feature request related to a problem? Please describe.

Hello!

To solve some workflow issues with torch I've developed a plugin that I'm storing in the same repository as the main project. I want all developers to have access to this tool for ease of use. However; there isn't any (documented?) way of putting a plugin into pyproject.toml, including configuration. I tried putting it in dev-dependencies, but since pdm runs in another environment it doesn't find the entrypoint.

Describe the solution you'd like

I'd to be able to specify plugins and their configuration in the project.toml. The config part is roughly how I've done it today; but I'm just digging out the key from the Project.pyproject data. The first part I don't think exists at all.

[tool.pdm]
plugins = [
    "pdm_plugin_torch @ file:///${PROJECT_ROOT}/plugins/pdm-plugin-torch",
]

[tool.pdm.plugin.pdm_plugin_torch]
torch-version = "torch==1.10.2"
lockfile = "torch.lock"
enable-cpu = true

enable-rocm = true
rocm-versions = ["4.2"]

enable-cuda = true
cuda-versions = ["cu111", "cu113"]

A potential format version that's a bit more structured:

[[tool.pdm.plugins]]
dependency = "pdm_plugin_torch @ file:///${PROJECT_ROOT}/plugins/pdm-plugin-torch"

[tool.pdm.plugins.configuration]
torch-version = "torch==1.10.2"
lockfile = "torch.lock"
enable-cpu = true

enable-rocm = true
rocm-versions = ["4.2"]

enable-cuda = true
cuda-versions = ["cu111", "cu113"]

[tgolsson]

In case anyone finds this between now and a proper solution, this works quite well to solve half of it.

[tool.pdm.scripts]
post_install = "pdm plugin add plugins/pdm-plugin-torch"

[frostming]

I doubt whether it is good behavior if your globally installed PDM changes from time to time depending on what project you are working on.

Maybe we can learn from lektor to install project-wide plugins in an isolated location and add that path to the PDM.

[tgolsson]

Yeah; I don't think installing them globally is necessarily good. On the other hand, if you already work in multiple repos and some need plugins, that issue already exists. I don't think this'd necessarily make it worse.

[tgolsson]

Also, we open-sourced the repo with the plugin yesterday here: https://github.com/EmbarkStudios/emote/tree/main/plugins/pdm-plugin-torch, in case anyone finds this while maybe searching for torch.

[noirbizarre]

I think this need to be splitted in 2 use-cases:

• plugins required to build the package (which in my humble opinion should simply be added to the build-system.requires entry if you don't want to break PEP517 compliance).
• suggested plugins for the remaining developper flow: I agree with @frostming that silent installation is not desirable but maybe something like VSCode plugins suggestion is acceptable: no silent install but a tool.pdm.[suggested-]plugins and maybe a plugin subcommand to install them explicitely (eg. pdm plugin install-suggested)

What do you think ?

[tgolsson]

The first use-case is not related to this issue, and I'm not sure how it'd be relevant to plugins.

I kind second is OK; but I'd prefer a virtual-environment based solution instead with automatic installs. I'd be fine with that as a temporary solution/stopgap, however. But it's not too far from hook workaround, and I can do the same with script too.

However: if we can combine that with a configuration solution I think we could get something good -- my structured example in the original post lets us combine configuration + dependency in one place. I.e., let's not focus only on installing - it's only half of the good solution. :-)

[franzhaas]

This is an awesome feature!

by adding
[tool.pdm] plugins = ["pdm-packer"]
and then running
pdm install --plugins

to my pyproject.toml, I am now able to use pdm pack from a zipapped pdm.

Thanks a lot!