Install dependency's extra based on my extra

[zyzzyxdonta]

• I have searched the issues of this repo and believe that this is not a duplicate.

Issue

Hi!

I'm trying to install a dependency's extra based on my own package's extra. E.g. I want to always install celery but when I specify -E redis, I want to install celery[redis]. I believe this case is not described in the documentation.

Here is a pyproject.toml where I tried some things but it didn't work out. In no combination of these lines would poetry install -E redis install redis.

An easy fix for my case would be to install redis itself as an optional dependency. However, this only works because celery[redis] is the same is celery plus redis (see here). An imagined case where it wouldn't work would be somepackage[caching] which installs somepackage plus somecache plus convenience_functions_for_somecache. Even if one would manually look up these dependencies, regular checks for changes would still be required.

So, is this possible with Poetry? If so, I think it would be great to see this described in the docs. If it is not possible, I would like to see it as a new feature because it feels to me like a fairly common case.

Thanks in advance!

[bacon1664]

We're facing this same problem with an internal dependency used in multiple micro-services. Solving it in the same way the OP works around celery and redis isn't an option as the internal dependency has multiple sets of optionals and that would require the dependents to know the dependency's dependencies and their individual version specs.

Is there a better way to solve this?

[dimbleby]

[tool.poetry.dependencies]
python = "^3.10"
celery = [
    { version = "^5.2.6" },
    { version = "^5.2.6", extras = ["redis"], optional = true },
]

[tool.poetry.extras]
redis = ["celery"]

works for me on master. As does the variation in your pyproject.toml that includes explicit markers = "extra == 'redis'".

You didn't say what version of poetry you're using, perhaps you could try again with the latest beta and confirm.

I expect that an MR improving the docs would be welcomed.

However it looks as though you had in fact found a good way to do this, but it wasn't working, so perhaps no need?

[zyzzyxdonta]

Thank you for your reply. I was using Poetry 1.1.13. With 1.2.0b2 it does indeed work. I'm looking forward to the release 😄

Is there/will there be a way to specify in the pyproject.toml that the project requires Poetry >= 1.2.0? The docs suggest you can specify the poetry_core version. But this doesn't really help, does it? In fact it seems you can just specify any version there, even ones that don't exist, and Poetry will run just fine without any error.

[dimbleby]

no

[dimbleby]

this can be closed

[igiloh-pinecone]

[tool.poetry.dependencies]
python = "^3.10"
celery = [
    { version = "^5.2.6" },
    { version = "^5.2.6", extras = ["redis"], optional = true },
]

[tool.poetry.extras]
redis = ["celery"]

@dimbleby this solution does seem to support poetry install correctly, but then fails after poetry build, when pip installing the built wheel.
The complementary solution which uses env markers does work for pip install of the built wheel, but doesn't work with poetry install, as indicated here.

Did anyone found a fully working solution for both poetry install and poetry build?

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.