Proposal: Allow multiple repositories to contain the same package

[edyoung]

If you have 2 repositories registered, and both of them contain a package Foo, you get an error when you run Install-Module Foo, saying that you have to pick a repository.

It would be better IMHO if this was permitted, and PowerShellGet automatically picked a repository when this happened. The main question is: when 2 repositories contain the requested module, how are ties broken - ie, which one do we pick?

Suggestion:

When Install-Module is run, we find all candidate repositories which contain a matching module, as at present. Then we order them using the method below to pick the preferred repo.

First, trusted vs untrusted is used. All trusted repositories are automatically higher ranked than all untrusted repos.

Second, introduce an explicit ordering for repositories, and a way for users to reorder them. The PSRepositoryInfo flag would have a Priority property added, which would be an int value, with 0 as lowest priority (last to be picked). This can be null, if not explicitly configured, which is treated as 0. Priorities need not be unique or consecutive.

Users can set the Priority property at Register-PSRepository time, and with Set-PSRepository -Priority. If not specified they get 0 priority.

Third, if 2 repositories have the same trust status and the same priority (including null) we pick them in alphabetical order.

Get-PSRepository would display the repos in priority order, NOT the order of the ordered table they are stored in (which is sorted by key). Additionally, it would show the Priority field by default.

This should not be a breaking change, since it only updates behavior for situations which would previously have caused an error.

Questions:

What if a repo with lower priority contains a module with higher version? Should that be taken into account? I think no.

Is it confusing to display an untrusted repo with Priority 999 and a trusted one with priority 1, and then choose the trusted one first?

Should the behavior of any other commands change?

What about package management, any changes needed there?

[alerickson]

I think this would be a nice addition and certainly make for a smoother experience.
It makes more sense to me to sort by priority, then trust, then alphabetically. If someone values priorities enough to explicitly set them for their repositories it seems like that should take precedence over whether the repo is trusted or not.

I don't think any other PSGet commands would change, but I'd have to look into package management.

[SydneyhSmith]

I agree that this is a good improvement to make. Something interesting when looking at other commands that could be effective is that Update-Module and Uninstall-Module work even with a module in multiple repositories i.e. they already use a hierarchy (based on version I think for Uninstall) while this wouldn't break Uninstall, it might make sense to extend the notion of priority of registers for consistency across commands

[edyoung]

Uninstall-Module doesn't care where the module came from; we record the repo a module was installed from in _psgetinfo.xml and Update-Module will only read from that repo when updating.

[kilasuit]

This is where $PSDefaultParameterValues come into play with `$PSDefaultParameterValues = @{'*-Module:Repository'='InternalPSGallery'}

either that or we need a Set-PSDefaultRepository function built which needs to also allow this default to be set at the machine level not just user level as per request in PowerShell/PowerShellGetv2#92

[awickham10]

I wouldn't mind taking a stab at at least partially implementing this if you need assistance. We have an immediate need for this type of functionality. We have two repositories (internal and public) and it would be nice to have our internal repository as a primary and then fallback to the public repo (eventually removing the public repo).

So we would make internal a trusted repo with public being untrusted and then just sort alphabetically. I imagine this would just require changing Get-PSRepository.ps1, Install-Module.ps1, and Install-Script.ps1. In Get-PSRepository I would return them in the correct order. In the Install* functions I would call Find* functions to find what repositories it lives in, and then sort based on the Get-PSRepository order.

Does that pretty much sum it up (other than the priority property)?

[edyoung]

That would be great! Nobody is looking at this to my knowledge. I feel a bit weird about using alphabetical order for sorting but can't think of a big problem that would result, and the priority thing makes it more complicated, so I am fine leaving it out - we could potentially add it later if we discover a good reason.

[awickham10]

Sounds good. I'm hoping to have something over this week. I think I have the Install-Module and Get-Module files modified, just need to come up with the tests to validate.

[SphenicPaul]

Another point of consideration (thought of since my raising of linked/related issue in PowerShell/PowerShellGetv2#310) is whether a user can access a PSRepository ... e.g. If a package exist on two repos, and one can be accessed an one can’t, then it takes it from the one it can access (potentially).

Also, if a PSModule is installed, can it auto-install ‘RequiredModules’ for that PSModule from any PSRepository available, rather than just the one original PSModule was installed from (although that PSRepository could be the first to be checked/searched).

Shout if you’ve any questions.

This seems like it would be likely to install the wrong package some of the time if someone is unaware of the behavior change, which could cause Bad Things... Maybe only auto-pick if the priority is set explicitly?

[testworksau]

Here's another point of consideration - when the same package is available on multiple repositories, but the available versions are different, which one should be chosen?

I'd say it should be the one in the Priority repository. The following error is currently displayed:

PS C:\Users\testworksau> Install-Module AWSPowerShell.NetCore
WARNING: 'AWSPowerShell.NetCore' matched module 'AWSPowerShell.NetCore/3.3.618.0' from provider: 'PowerShellGet', repository 'PSGallery'.
WARNING: 'AWSPowerShell.NetCore' matched module 'AWSPowerShell.NetCore/3.3.590' from provider: 'PowerShellGet', repository 'artifactory'.
PackageManagement\Install-Package : Unable to install, multiple modules matched 'AWSPowerShell.NetCore'. Please specify a single -Repository.
At C:\program files\powershell\6\Modules\PowerShellGet\PSModule.psm1:9508 char:21

[SphenicPaul]

Hi.

I could have missed something in respect of this, but is there even the concept of PSRepository “priority”? If so, where is this referenced/assigned?

If you have differing PSModule versions, would you not just take the latest version (akin to the logic used against a single PSRepository)?

With multiple PSRepositories, when no explicit “RequiredVersion” is provided, there is the downside that all PSRepositories have to be searched to get the available, latest versions in each one. If a “RequiredVersion” is provided, then searching repositories can stop once that version is found in one of them.

[alerickson]

Yup, there's no "priority" right now... the latest version is always the package that's picked. We're working on a re-write of PowerShellGet that will include an option to set repository priorities, which will help to address the issue of searching through multiple repositories to find the latest version.

[testworksau]

Yup, there's no "priority" right now... the latest version is always the package that's picked.

I'm confused by this statement. My previous comment shows that it did not select the latest package - it simply failed.

[kilasuit]

@testworksau PowerShellGet currently isn't built to allow a user to install from just any repository. It expects you to tell it which one you want to install from if there are multiple registered.

So you will need to tell it which one with -Repository artifactory for example or you can add it to your $PSDefaultParameterValues so that it will always install from that one

[testworksau]

@testworksau PowerShellGet currently isn't built to allow a user to install from just any repository. It expects you to tell it which one you want to install from if there are multiple registered.

Yes, that's been my experience, but the statement in the comment above my last one talks about picking the latest version. Perhaps I misinterpreted that as we were both querying package selection in the context of using multiple repositories.

So you will need to tell it which one with -Repository artifactory for example or you can add it to your $PSDefaultParameterValues so that it will always install from that one

I didn't know about this - that may be useful - thanks!