[question] Documenting precedence behavior of multiple repos

[duffsterlp]

What is your question?

Hello conan community!

I am using conan 1.X (we are in the process of transitioning to conan 2 but it's a long transition).

I apologize if I've missed this in the documentation, but I have some questions about precedence ordering when using multiple repos. Here are my specific questions in the context of configuring multiple repos using conan remote add.

• If you specify a version range of a component and the first repo in the list has a particular version of that component but subsequent repos in the list have newer versions, then which version gets returned?
• If you have two identical versions of a component in multiple repos and the second repo in the list has a newer recipe revision, which revision ends up being used (if you don't explicitly point to a particular revision)
• Same as the above question for package revisions

Are these behaviors documented in the conan documentation?

Thanks in advance!

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide (went back and read this guide after creating the issue)

[memsharded]

Hi @duffsterlp

Thanks for your question.

If you specify a version range of a component and the first repo in the list has a particular version of that component but subsequent repos in the list have newer versions, then which version gets returned?

It retrieves the first occurrence of the first version that fits in the range, starting from the Conan cache (if it is in the cache, it will not even look for a more modern one in remotes)

If you want to ensure that it resolves to the latest version in any of the remotes, using the --update argument does that. The trade-off is that it is slower, as it has to check in every configured remote

If you have two identical versions of a component in multiple repos and the second repo in the list has a newer recipe revision, which revision ends up being used (if you don't explicitly point to a particular revision)

Same applies for recipe revisions than for regular versions (recipe revision is basically an automatic version mechanism), so it will resolve to the first found recipe-revision unless --update is provided, in that case, it will check for the latest recipe revision in all remotes

Same as the above question for package revisions

It is important to be aware that having multiple package revisions for the same binary are discouraged, because it is an indicator of a flaw in the model or in the process. The idea is that:

• The version + recipe-revision accounts for any variation in input sources, including package definition
• The package_id accounts for any variation in configuration, including settings, options and dependencies

With those two things, everything should be fully defined for every binary, so a given version + recipe-revision + package_id should only need to build its binary just once. Rebuilding a new binary for that same inputs, that will create a new package-revision is unnecessary and not recommended. If there is something that actually changes the binary identity, it should be made part of the package_id information via settings, options, conf, or dependencies package_id modes.

Said that, same rules above applies for package-revisions, the first one found in the general case, the latest one if using --update

[duffsterlp]

@memsharded Thank you. This exactly answers my question.

Are these behaviors documented? If not, do you think they should be?

For example, I see the following description for the update flag in conan install:

  -u [UPDATE], --update [UPDATE]
                        Will install newer versions and/or revisions in the
                        local cache for the given reference name, or all
                        references in the graph if no argument is supplied.
                        When using version ranges, it will install the latest
                        version that satisfies the range. It will update to
                        the latest revision for the resolved version range.

I can see that it could be implied by "latest version" or "latest revision" that this is friendly to multiple remotes - but there is nothing there that explicitly mentions multiple remotes.

Thanks!

(Also it's good to know about the package revisions not being desired behavior - They do change sporadically in our environment but it's not part of our design - it's more to do with forced rebuilds where some files in the package change due to an environment change or due to timestamps being part of the file format)

[memsharded]

Moved this to the docs repo to try to add some further details there. Thanks!

[duffsterlp]

@memsharded Thanks for moving this to the docs area.

One further question - are all of the behaviors you described the same if you have multiple local artifactory repos fronted by a single virtual repo? Say you have virt-repo that houses repoA and repoB and the conan config only points to virt-repo. Does it still iterate through the repos behind the scenes in the same way described when referencing multiple physical repos in the conan config? Is that still conditional on whether the -u flag is used? Is this behavior consistent between conan 1 and conan 2?

[memsharded]

One further question - are all of the behaviors you described the same if you have multiple local artifactory repos fronted by a single virtual repo? Say you have virt-repo that houses repoA and repoB and the conan config only points to virt-repo. Does it still iterate through the repos behind the scenes in the same way described when referencing multiple physical repos in the conan config? Is that still conditional on whether the -u flag is used? Is this behavior consistent between conan 1 and conan 2?

Artifactory virtual repos haven't changed at all, the behavior is exactly the same for Conan 1 and Conan 2.
The resolution against a virtual repo is not affected by the --update argument, that is just an argument for the client to go and check the servers, but from the client point of view there is no distinction between a remote "local" repo or a remote "virtual" repo.

I think internally the virtual repos do iterate their aggregated local repos looking for resolution of dependencies, but I am not sure about how it works for updates, as I understand that virtual repos also have some caching capabilities, so it might depend on that and how it is configured in the server side. This might be a question more for Artifactory support, I am afraid I don't know enough about those details.

[duffsterlp]

One further question - are all of the behaviors you described the same if you have multiple local artifactory repos fronted by a single virtual repo? Say you have virt-repo that houses repoA and repoB and the conan config only points to virt-repo. Does it still iterate through the repos behind the scenes in the same way described when referencing multiple physical repos in the conan config? Is that still conditional on whether the -u flag is used? Is this behavior consistent between conan 1 and conan 2?

Artifactory virtual repos haven't changed at all, the behavior is exactly the same for Conan 1 and Conan 2. The resolution against a virtual repo is not affected by the --update argument, that is just an argument for the client to go and check the servers, but from the client point of view there is no distinction between a remote "local" repo or a remote "virtual" repo.

I think internally the virtual repos do iterate their aggregated local repos looking for resolution of dependencies, but I am not sure about how it works for updates, as I understand that virtual repos also have some caching capabilities, so it might depend on that and how it is configured in the server side. This might be a question more for Artifactory support, I am afraid I don't know enough about those details.

Thanks for the information @memsharded