Support for combining sources/indexes

[samskiter]

Issue Kind

Brand new capability

Description

Indexes such as piwheels exist to provide builds of python packages for additional platforms/architectures that are not provided by PyPI.

When using these with Pip, it is common to add them as an additional 'source' for wheels. Pip will search all provided indexes until it finds a wheel that it can install.

Poetry, on the other hand, will only record a single source per package when creating the lock file. This causes 2 issues:

1. Cross-architecture installations
   • It is impossible to generate a lock file that can be used on two architectures, where one is provided for by piwheels and the other is provided for by pypi
2. Complex single architecture installations
   • Some packages are found on one index, but happen not to be installable on that architecture - so a single architecture needs to pull from 2 indexes, but some those packages are available on both.

Impact

Poetry is largely unusable on Raspberry Pi:

#9145

Workarounds

Using markers and iterating

[dimbleby]

we don't need both this and #9145, please close one or the other

[neersighted]

I haven't checked yet to see if this is a duplicate; I suspect it is. That being said, the way Poetry treats each source as distinct instead of blending them together is a big differentiator from pip, and a major feature for most users.

I do think there is potentially some interesting possibilities to create a 'combined' source from one or more repo definitions (and indeed, perhaps some form of ordering makes sense there as well?) -- certainly with a lot of the sources refactoring that has happened recently (as well as PEP 658 support), this is a lot more practical than it was in the past.

It will also introduce potential complicates with e.g. metadata consistency validation and other features we want to add, so I think we should tread carefully in any case.

And if anyone can check if this issue is a duplicate/we just need to update discussion in an existing issue, it would be appreciated!