feat(gomod): use git host rules as authentication for gosum updates

[Shegox]

Changes:

This PR adds all known GitHub, GitLab (and tbd) credentials it can find in the host rules as Git Environment variables for the gosum update as outlined in #7361 (comment)

• The list of registries to provide authentation from should not be derived from registryUrls
• Instead, the list should come from hostRules
• Host rules could have hostType=go but we should probably also automatically do any with hostType=github too, maybe also hostType=gitlab?
• Need to think about if we need to distinguish between api.github.com and github.com
• Need to be clear that it relates to artifacts updating (go.sum, vendored modules, etc) and not to Renovate's own lookup phase

As of today it looks for all hostRules matching GitHub and GitLab. A custom hostType=go is not yet supported (but would be only 3 more lines). It currently uses hostRule.matchHost to construct the GitUrl it should use the credentials for. This allows it to use paths as well.

Open Topics/Questions:

• Should we add a custom hostType=go for it? // not for now
• Should we support all the other git platforms out of the box as well (bitbucket, gittea)? Is there a generic hosttype git I can use? // yes all platform + without hostType
• Should we use hostRule.matchHost (supports paths) or hostRule.resolvedHost and construct the http(s) url from there? // hostRule.matchHost

To be done:

• Docs needs to be updated
• Run against a real repository

Context:

fixes #7361
follow up to #11077

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[Shegox]

@rarkins I still have to update the docs and do some further testing with some of my own repositories, but if you have time I would like to ask you if you can please take already a look at the implementation and see if that is what you imagined. Many thanks in advance.

[rarkins]

Some questions:

• Is there any harm in adding all hosts, even if they have nothing to do with Go (e.g. could be a Rubygems registry)?
• Do we need to be "platform aware" (e.g. github, gitlab, etc) or if we do all hosts then will we have enough rules in place anyway?

[Shegox]

Is there any harm in adding all hosts, even if they have nothing to do with Go (e.g. could be a Rubygems registry)?
Not really as long as the hostname don't overlap. E.g.:

        {
            "matchHost": "github.enterprise.com",
            "token": "invalid"
        },
        {
            "matchHost": "github.enterprise.com/test",
            "token": "token"
        },

Might cause problems when accessing go modules under github.enterprise.com/test. I think however this is a very rare problem.

Do we need to be "platform aware" (e.g. github, gitlab, etc) or if we do all hosts then will we have enough rules in place anyway?

At the end of the day go get -d only uses git, so we don't need to be platform aware in any way as long as git clone works against the system. (There are other supported platform for go get as well, but git should be the majority).

[rarkins]

Not really as long as the hostname don't overlap. E.g.:

We could perhaps do an intelligent de-duplication to avoid that problem?

[Shegox]

I just tested it again and I had an error in my previous testing (the github.enterprise.com with invalid) token affected other things as well causing problems.
Using works as expected. Even through the rule for github.enterprise.com is added to Git the other more precise hostname overrules it github.enterprise.com/test.

        {
            "matchHost": "github.enterprise.com",
            "token": "invalid",
            "hostType":"npm"
        },
        {
            "matchHost": "github.enterprise.com/test",
            "token": "token"
        },

I think this should be generally fine. The only other option I see would be to narrow it down to the 6 Git Platforms that are supported plus rules without hostType.

[viceice]

i think we should only use specific known host types plus new gomod for custom unknown hosts

[rarkins]

So known git platforms only?

[Shegox]

I would suggest, in this order

1. Known git-platforms (github, gitlab, azure devops, bitbucket, gittea)
2. A generic new git type (maybe?)
3. Rules without hostType

And adding it only if it is a different matchHost as an additional GIT_CONFIG.

        {
            "matchHost": "github.enterprise.com/test",
            "token": "token",
            "hostType":"github"
        },
        {
            "matchHost": "github.enterprise.com/test",
            "token": "invalid"
        },

Should result it token being token.

[rarkins]

This could be useful for more managers than just gomod, too

[renovate-release]

🎉 This PR is included in version 28.19.0 🎉

The release is available on:

• GitHub release
• 28.19.0

Your semantic-release bot 📦🚀