fix(datasource/sbt-package): Fallbacks to maven when no sbt directory listing available

[fernandomora]

Changes

Fallbacks sbt datasources to maven datasource (which fetches versions based in maven-metadata.xml) if directory listing first approach is not successful.

Context

#18589

Using sbt datasource is not compatible with any maven repository that does not allow listing organization folder contents

sbt datasource tries to list all organization packages for the given package inside the repository to look for available versions of the same package with different suffixes.

That approach is unsuitable in repositories that doesn't allow directory listing, and only suitable option I see would be to fallback to maven datasource that checks maven-metadata.xml file with available versions for same package.

Closes #18589

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[fernandomora]

Tested in this repository before the changes: https://github.com/fernandomora/renovate-sbt-test

https://app.renovatebot.com/dashboard#github/fernandomora/renovate-sbt-test/865267319

DEBUG: No versions found for com.intenthq:scala-secret_2.13 in 2 repositories
DEBUG: GET https://maven.pkg.github.com/intenthq/_/com/intenthq/ = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=422 retryCount=0, duration=113)
INFO: Unknown HTTP download error
{
  "failedUrl": "https://maven.pkg.github.com/intenthq/_/com/intenthq/",
  "err": {
    "name": "HTTPError",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "timings": {
      "start": 1666804602273,
      "socket": 1666804602273,
      "lookup": 1666804602285,
      "connect": 1666804602285,
      "secureConnect": 1666804602296,
      "upload": 1666804602296,
      "response": 1666804602385,
      "end": 1666804602386,
      "phases": {
        "wait": 0,
        "dns": 12,
        "tcp": 0,
        "tls": 11,
        "request": 0,
        "firstByte": 89,
        "download": 1,
        "total": 113
      }
    },
    "message": "Response code 422 (Unprocessable Entity)",
    "stack": "HTTPError: Response code 422 (Unprocessable Entity)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
    "options": {
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "authorization": "***********",
        "accept-encoding": "gzip, deflate, br"
      },
      "url": "https://maven.pkg.github.com/intenthq/_/com/intenthq/",
      "hostType": "sbt",
      "username": "",
      "password": "",
      "method": "GET",
      "http2": false
    },
    "response": {
      "statusCode": 422,
      "statusMessage": "Unprocessable Entity",
      "body": "invalid path for maven file\n",
      "headers": {
        "access-control-allow-methods": "GET, HEAD, OPTIONS",
        "access-control-allow-origin": "*",
        "content-security-policy": "default-src 'none';",
        "content-type": "text/plain; charset=utf-8",
        "server": "GitHub Registry",
        "strict-transport-security": "max-age=31536000;",
        "x-content-type-options": "nosniff",
        "x-frame-options": "DENY",
        "x-xss-protection": "1; mode=block",
        "date": "Wed, 26 Oct 2022 17:16:42 GMT",
        "content-length": "28",
        "x-github-request-id": "8A34:6B20:5F2B48:6BB8AA:63596B7A",
        "connection": "close"
      },
      "httpVersion": "1.1",
      "retryCount": 0
    }
  }
}
...
DEBUG: No versions found for com.intenthq:scala-secret_2.13 in 2 repositories
DEBUG: Failed to look up sbt-package dependency com.intenthq:scala-secret (com.intenthq:scala-secret)(packageFile="build.sbt", dependency="com.intenthq:scala-secret")
DEBUG: Found package lookup warnings
{
  "warnings": [
    "Failed to look up sbt-package dependency com.intenthq:scala-secret"
  ],
  "warningFiles": [
    "build.sbt"
  ]
}

[fernandomora]

Tested in this repository: fernandomora/renovate-sbt-test

Logs after the changes running it locally:

DEBUG: No versions discovered for com.intenthq:scala-secret_2.13 listing organization root package folder, fallback to maven datasource for version discovery (repository=fernandomora/renovate-sbt-test)
DEBUG: Looking up com.intenthq:scala-secret_2.13 in repository https://maven.pkg.github.com/intenthq/_/ (repository=fernandomora/renovate-sbt-test)
DEBUG: Found 2 new releases for com.intenthq:scala-secret_2.13 in repository https://maven.pkg.github.com/intenthq/_/ (repository=fernandomora/renovate-sbt-test)
DEBUG: PackageFiles.add() - Package file saved for branch (repository=fernandomora/renovate-sbt-test, baseBranch=main)
DEBUG: Package releases lookups complete (repository=fernandomora/renovate-sbt-test, baseBranch=main)
DEBUG: branchifyUpgrades (repository=fernandomora/renovate-sbt-test)
DEBUG: 2 flattened updates found: scala, com.intenthq:scala-secret (repository=fernandomora/renovate-sbt-test)
...
 INFO: Branch created (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
       "commitSha": "4ecf195079b7d38cd6fc954ad366c7e134b51135"
DEBUG: Ensuring PR (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
DEBUG: There are 0 errors and 0 warnings (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
DEBUG: getBranchPr(renovate/com.intenthq-scala-secret-1.x) (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
DEBUG: findPr(renovate/com.intenthq-scala-secret-1.x, undefined, open) (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
DEBUG: findPr(renovate/com.intenthq-scala-secret-1.x, undefined, closed) (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
DEBUG: Creating PR (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
       "prTitle": "Update dependency com.intenthq:scala-secret to v1.1.0"
DEBUG: Creating PR (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
       "title": "Update dependency com.intenthq:scala-secret to v1.1.0",
       "head": "fernandomora:renovate/com.intenthq-scala-secret-1.x",
       "base": "main",
       "draft": false
DEBUG: PR created (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
       "pr": 5,
       "draft": false
DEBUG: Adding labels 'dependencies' to #5 (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
 INFO: PR created (repository=fernandomora/renovate-sbt-test, branch=renovate/com.intenthq-scala-secret-1.x)
       "pr": 5,
       "prTitle": "Update dependency com.intenthq:scala-secret to v1.1.0"

Successfully created PR: fernandomora/renovate-sbt-test#5

[zharinov]

LGTM, just one minor suggestion

[viceice]

waiting for @zharinov feedback

[viceice]

🤷‍♂️

[renovate-release]

🎉 This PR is included in version 34.21.1 🎉

The release is available on:

• GitHub release
• 34.21.1

Your semantic-release bot 📦🚀