Secrets are not applied to hostRules if both are configured in global config

[rarkins]

Discussed in #16200

^{Originally posted by michael-sicpa June 22, 2022}

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

32.91.2

Please select which platform you are using if self-hosting.

Bitbucket Server

If you're self-hosting Renovate, tell us what version of the platform you run.

NA

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

Hi,
In my config file I have the following to use secrets, as explained in the documentation:

"hostRules": [
    {
      "hostType": "maven",
      "matchHost": "https://internal-repo/maven-repos",
      "username": "serviceAccountRenovate",
      "password": "{{ secrets.ARTIFACTORY_PASSWORD }}"
    }
],
"repositories": [
    "XX/YY"
  ]
...

Unfortunately, this doesn't work, it never get replaced, or if I put a non-existing secret, it doesn't exit with an error.
After some debugging in the code, I found the issue.
At this line you reset the hostRules, then you pass this to config to "applySecretsToConfig" method, so obviously it will never replace any secrets here.

Can you fix it?
I saw you have a unit test with hostRules and secret, but it calls directly "applySecretsToConfig", so you don't have this issue.
Am I doing something wrong?
Thank you in advance for your help,

Relevant debug logs

Logs

DEBUG: Using RE2 as regex engine
DEBUG: Parsing configs
DEBUG: Checking for config file in /Users/XXX/Projects/renovate/renovate-config/renovate-config-env.json
DEBUG: File config
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "platform": "bitbucket-server",
         "endpoint": "https://git.XXX.com/",
         "username": "svcteamRenovate",
         "gitAuthor": "Renovate Bot <renovate.bot@XXX.com>",
         "hostRules": [
           {
             "hostType": "maven",
             "matchHost": "https://artifactory.XXX.com/maven-repos",
             "username": "svcteamRenovate",
             "password": "***********"
           }
         ],
         "dryRun": "full",
         "printConfig": false,
         "requireConfig": "optional",
         "onboarding": false,
         "branchPrefix": "updates/",
         "prFooter": "This PR has been generated by [Renovate BOT].",
         "repositories": ["team/bff-support"],
         "baseBranches": ["develop"],
         "enabledManagers": ["maven"],
         "helmv3": {
           "registryAliases": {"XXX": "https://artifactory.XXX.com/helm-repos"}
         },
         "packageRules": [
           {
             "matchManagers": ["maven"],
             "excludePackagePatterns": ["com.XXX.team.*"],
             "enabled": false
           },
           {
             "matchManagers": ["maven"],
             "registryUrls": ["https://artifactory.XXX.com/maven-repos"],
             "versioning": "regex:^(?<major>\\d+)(\\.(?<minor>\\d+))?(\\.(?<patch>\\d+))?(?<prerelease>-SNAPSHOT)?$",
             "matchPackagePatterns": ["com.XXX.team.*"]
           },
           {
             "matchUpdateTypes": ["minor", "patch"],
             "groupName": "team minor dependencies"
           }
         ]
       }
DEBUG: CLI config
       "config": {}
DEBUG: Env config
       "config": {"hostRules": [], "password": "***********"}
DEBUG: Combined config
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "platform": "bitbucket-server",
         "endpoint": "https://git.XXX.com/",
         "username": "svcteamRenovate",
         "gitAuthor": "Renovate Bot <renovate.bot@XXX.com>",
         "hostRules": [
           {
             "hostType": "maven",
             "matchHost": "https://artifactory.XXX.com/maven-repos",
             "username": "svcteamRenovate",
             "password": "***********"
           }
         ],
         "dryRun": "full",
         "printConfig": false,
         "requireConfig": "optional",
         "onboarding": false,
         "branchPrefix": "updates/",
         "prFooter": "This PR has been generated by [Renovate BOT].",
         "repositories": ["team/bff-support"],
         "baseBranches": ["develop"],
         "enabledManagers": ["maven"],
         "helmv3": {
           "registryAliases": {"XXX": "https://artifactory.XXX.com/helm-repos"}
         },
         "packageRules": [
           {
             "matchManagers": ["maven"],
             "excludePackagePatterns": ["com.XXX.team.*"],
             "enabled": false
           },
           {
             "matchManagers": ["maven"],
             "registryUrls": ["https://artifactory.XXX.com/maven-repos"],
             "versioning": "regex:^(?<major>\\d+)(\\.(?<minor>\\d+))?(\\.(?<patch>\\d+))?(?<prerelease>-SNAPSHOT)?$",
             "matchPackagePatterns": ["com.XXX.team.*"]
           },
           {
             "matchUpdateTypes": ["minor", "patch"],
             "groupName": "team minor dependencies"
           }
         ],
         "password": "***********"
       }
DEBUG: Adding trailing slash to endpoint
DEBUG: Found valid git version: 2.36.1
DEBUG: Using configured gitAuthor (Renovate Bot <renovate.bot@XXX.com>)
DEBUG: Adding password authentication for git.XXX.com to hostRules
DEBUG: Using baseDir: /var/folders/9z/94mg7rkd7x97c1fvzjrn3mm97x4kjw/T/renovate
DEBUG: Using cacheDir: /var/folders/9z/94mg7rkd7x97c1fvzjrn3mm97x4kjw/T/renovate/cache
DEBUG: Initializing Renovate internal cache into /var/folders/9z/94mg7rkd7x97c1fvzjrn3mm97x4kjw/T/renovate/cache/renovate/renovate-cache-v1
DEBUG: Commits limit = null
DEBUG: Setting global hostRules
DEBUG: Adding password authentication for https://artifactory.XXX.com/maven-repos to hostRules
DEBUG: Adding password authentication for git.XXX.com to hostRules
DEBUG: validatePresets()
DEBUG: Reinitializing hostRules for repo
DEBUG: Clearing hostRules
DEBUG: Adding password authentication for https://artifactory.XXX.com/maven-repos to hostRules
DEBUG: Adding password authentication for git.XXX.com to hostRules
 INFO: Repository started (repository=team/bff-support)
       "renovateVersion": "32.91.2"
DEBUG: Using localDir: /var/folders/9z/94mg7rkd7x97c1fvzjrn3mm97x4kjw/T/renovate/repos/bitbucket-server/team/bff-support (repository=team/bff-support)
DEBUG: PackageFiles.clear() - Package files deleted (repository=team/bff-support)
       "baseBranches": []
DEBUG: resetMemCache() (repository=team/bff-support)
DEBUG: initRepo("{
         "repository": "team/bff-support"
       }") (repository=team/bff-support)
(node:36561) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
DEBUG: team/bff-support owner = team (repository=team/bff-support)
DEBUG: Resetting npmrc (repository=team/bff-support)
DEBUG: detectSemanticCommits() (repository=team/bff-support)
DEBUG: Initializing git repository into /var/folders/9z/94mg7rkd7x97c1fvzjrn3mm97x4kjw/T/renovate/repos/bitbucket-server/team/bff-support (repository=team/bff-support)
DEBUG: Performing full clone (repository=team/bff-support)
DEBUG: git clone completed (repository=team/bff-support)
       "durationMs": 518
DEBUG: latest repository commit (repository=team/bff-support)
       "latestCommit": {
         "hash": "80ee0c77a0e2fc7580caa9edff1630ccd13af75c",
         "date": "2021-04-09T13:01:07+00:00",
         "message": "Update version to 1.2.1-SNAPSHOT",
         "refs": "HEAD -> develop, origin/develop, origin/HEAD",
         "body": "",
         "author_name": "svcPsdGit (Jenkins)",
         "author_email": "svcPsdGit@XXX.com"
       }
DEBUG: getCommitMessages (repository=team/bff-support)
DEBUG: Semantic commits detection: atom (repository=team/bff-support)
DEBUG: No semantic commits detected (repository=team/bff-support)
DEBUG: checkOnboarding() (repository=team/bff-support)
DEBUG: isOnboarded() (repository=team/bff-support)
DEBUG: Repo is onboarded (repository=team/bff-support)
DEBUG: No renovate config file found (repository=team/bff-support)
DEBUG: migrateAndValidate() (repository=team/bff-support)
DEBUG: No config migration necessary (repository=team/bff-support)
DEBUG: massaged config (repository=team/bff-support)
       "config": {}
DEBUG: migrated config (repository=team/bff-support)
       "config": {}
DEBUG: Found repo ignorePaths (repository=team/bff-support)
       "ignorePaths": ["**/node_modules/**", "**/bower_components/**"]
DEBUG: getVulnerabilityAlerts() (repository=team/bff-support)
DEBUG: No vulnerability alerts found (repository=team/bff-support)
DEBUG: baseBranches (repository=team/bff-support)
       "baseBranches": ["develop"]
DEBUG: baseBranch: develop (repository=team/bff-support)
DEBUG: extract() (repository=team/bff-support)
DEBUG: Setting current branch to develop (repository=team/bff-support)
DEBUG: latest commit (repository=team/bff-support)
       "branchName": "develop",
       "latestCommitDate": "2021-04-09T13:01:07+00:00"
DEBUG: Applying enabledManagers filtering (repository=team/bff-support)
DEBUG: Using file match: (^|/|\.)pom\.xml$ for manager maven (repository=team/bff-support)
DEBUG: Using file match: ^(((\.mvn)|(\.m2))/)?settings\.xml$ for manager maven (repository=team/bff-support)
DEBUG: Matched 1 file(s) for manager maven: pom.xml (repository=team/bff-support)
DEBUG: Found maven package files (repository=team/bff-support)
DEBUG: Found 1 package file(s) (repository=team/bff-support)
 INFO: Dependency extraction complete (repository=team/bff-support)
       "baseBranch": "develop",
       "stats": {
         "managers": {"maven": {"fileCount": 1, "depCount": 1}},
         "total": {"fileCount": 1, "depCount": 1}
       }
DEBUG: baseBranch: develop (repository=team/bff-support)
DEBUG: Looking up com.XXX.team:team-parent in repository https://artifactory.XXX.com/maven-repos/ (repository=team/bff-support)
DEBUG: Dependency lookup unauthorized. Please add authentication with a hostRule (repository=team/bff-support)
       "failedUrl": "https://artifactory.XXX.com/maven-repos/com/XXX/team/team-parent/maven-metadata.xml"
DEBUG: Content is not found for Maven url (repository=team/bff-support)
       "url": "https://artifactory.XXX.com/maven-repos/com/XXX/team/team-parent/maven-metadata.xml",
       "statusCode": undefined
DEBUG: Failed to look up dependency com.XXX.team:team-parent (repository=team/bff-support, packageFile=pom.xml, dependency=com.XXX.team:team-parent)
DEBUG: PackageFiles.add() - Package file saved for branch (repository=team/bff-support)
       "baseBranch": "develop"
DEBUG: Package releases lookups complete (repository=team/bff-support)
       "baseBranch": "develop"
DEBUG: branchifyUpgrades (repository=team/bff-support)
DEBUG: 0 flattened updates found:  (repository=team/bff-support)
DEBUG: Returning 0 branch(es) (repository=team/bff-support)
DEBUG: config.repoIsOnboarded=true (repository=team/bff-support)
DEBUG: packageFiles with updates (repository=team/bff-support)
       "config": {
         "maven": [
           {
             "datasource": "maven",
             "packageFile": "pom.xml",
             "deps": [
               {
                 "datasource": "maven",
                 "depName": "com.XXX.team:team-parent",
                 "currentValue": "2.5.1",
                 "fileReplacePosition": 401,
                 "registryUrls": ["https://repo.maven.apache.org/maven2"],
                 "depType": "parent",
                 "depIndex": 0,
                 "updates": [],
                 "warnings": [
                   {
                     "topic": "com.XXX.team:team-parent",
                     "message": "Failed to look up dependency com.XXX.team:team-parent"
                   }
                 ],
                 "versioning": "regex:^(?<major>\\d+)(\\.(?<minor>\\d+))?(\\.(?<patch>\\d+))?(?<prerelease>-SNAPSHOT)?$"
               }
             ],
             "parent": "../pom.xml",
             "packageFileVersion": "1.2.1-SNAPSHOT"
           }
         ]
       }
DEBUG: processRepo() (repository=team/bff-support)
DEBUG: Processing 0 branches:  (repository=team/bff-support)
DEBUG: Calculated maximum PRs remaining this run (repository=team/bff-support)
       "prsRemaining": 99
DEBUG: PullRequests limit = 99 (repository=team/bff-support)
DEBUG: Calculated maximum branches remaining this run (repository=team/bff-support)
       "branchesRemaining": 99
DEBUG: Branches limit = 99 (repository=team/bff-support)
 INFO: DRY-RUN: Would close Dependency Dashboard (repository=team/bff-support)
       "title": "Dependency Dashboard"
DEBUG: Removing any stale branches (repository=team/bff-support)
DEBUG: config.repoIsOnboarded=true (repository=team/bff-support)
DEBUG: No renovate branches found (repository=team/bff-support)
DEBUG: ensureIssueClosing(Action Required: Fix Renovate Configuration) (repository=team/bff-support)
DEBUG: PackageFiles.clear() - Package files deleted (repository=team/bff-support)
       "baseBranches": ["develop"]
DEBUG: getPrList() (repository=team/bff-support)
DEBUG: Retrieved Pull Requests (repository=team/bff-support)
       "length": 0
DEBUG: Renovate repository PR statistics (repository=team/bff-support)
       "stats": {"total": 0, "open": 0, "closed": 0, "merged": 0}
DEBUG: Repository result: done, status: onboarded, enabled: true, onboarded: true (repository=team/bff-support)
DEBUG: Repository timing splits (milliseconds) (repository=team/bff-support)
       "splits": {"init": 1314, "extract": 200, "lookup": 228, "update": 2},
       "total": 1839
DEBUG: http statistics (repository=team/bff-support)
       "urls": {
         "https://artifactory.XXX.com/maven-repos/com/XXX/team/team-parent/maven-metadata.xml (GET,401)": 1,
         "https://git.XXX.com/./rest/api/1.0/projects/team/repos/bff-support/pull-requests (GET,200)": 1
       },
       "hostStats": {
         "artifactory.XXX.com": {
           "requestCount": 1,
           "requestAvgMs": 199,
           "queueAvgMs": 0
         },
         "git.XXX.com": {"requestCount": 1, "requestAvgMs": 69, "queueAvgMs": 0}
       },
       "totalRequests": 2
 INFO: Repository finished (repository=team/bff-support)
       "durationMs": 1839
DEBUG: Renovate exiting

Have you created a minimal reproduction repository?

No reproduction repository

[renovate-release]

🎉 This issue has been resolved in version 32.97.0 🎉

The release is available on:

• GitHub release
• 32.97.0

Your semantic-release bot 📦🚀