Renovate fails to update pre-commit hook from codeberg.org repository

[wwuck]

How are you running Renovate?

Self-hosted Renovate CLI

Which platform you running Renovate on?

GitLab (.com or self-hosted)

Which version of Renovate are you using?

a93daa4

Please tell us more about your question or problem

I have a pre-commit config referencing a hook on codeberg.org. Renovate fails to detect the repository on codeberg.org and ignores it.

Reproduction repository: https://github.com/wwuck/renovate-pre-commit-codeberg

Does renovate need to add support for codeberg.org as a "Supported datasource" of type "Forgejo Tags" for the pre-commit manager?

Logs (if relevant)

Logs

DEBUG: Matched 1 file(s) for manager pre-commit: .pre-commit-config.yaml (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Matched 1 file(s) for manager renovate-config: renovate.json (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Found version v0.6.37 (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Provided hostname does not match any hostRules. Ignoring (repository=https://codeberg.org/gherynos/pre-commit-java)
       "hostUrl": "https://codeberg.org"
DEBUG: manager extract durations (ms) (repository=wwuck/renovate-pre-commit-codeberg)
       "managers": {"pre-commit": 8, "renovate-config": 9}
DEBUG: Found pre-commit package files (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Found 1 package file(s) (repository=wwuck/renovate-pre-commit-codeberg)
 INFO: Dependency extraction complete (repository=wwuck/renovate-pre-commit-codeberg, baseBranch=main)
       "stats": {
         "managers": {"pre-commit": {"fileCount": 1, "depCount": 1}},
         "total": {"fileCount": 1, "depCount": 1}
       }
DEBUG: PackageFiles.add() - Package file saved for base branch (repository=wwuck/renovate-pre-commit-codeberg, baseBranch=main)
DEBUG: Package releases lookups complete (repository=wwuck/renovate-pre-commit-codeberg, baseBranch=main)
DEBUG: Repository libYears (repository=wwuck/renovate-pre-commit-codeberg)
       "libYears": {"managers": {"pre-commit": 0}, "total": 0},
       "dependencyStatus": {"outdated": 0, "total": 1}
DEBUG: branchifyUpgrades (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: detectSemanticCommits() (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: getCommitMessages (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: semanticCommits: score=1 (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: semanticCommits: enabled (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: 0 flattened updates found:  (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Returning 0 branch(es) (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: config.repoIsOnboarded=true (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: packageFiles with updates (repository=wwuck/renovate-pre-commit-codeberg, baseBranch=main)
       "config": {
         "pre-commit": [
           {
             "deps": [
               {
                 "skipReason": "unknown-registry",
                 "registryUrls": ["codeberg.org"],
                 "depName": "gherynos/pre-commit-java",
                 "depType": "repository",
                 "packageName": "gherynos/pre-commit-java",
                 "currentValue": "v0.6.37",
                 "updates": []
               }
             ],
             "packageFile": ".pre-commit-config.yaml"
           }
         ]
       }

[viceice]

yes,

renovate/lib/modules/manager/pre-commit/extract.ts

Lines 46 to 55 in 71bb1e2

	if (detectPlatform(repository) === 'gitlab') {
	logger.debug(
	{ repository, hostname },
	'Found gitlab dependency with custom registryUrl',
	);
	return {
	datasource: GitlabTagsDatasource.id,
	registryUrls: [`https://${hostname}`],
	};
	}

should be something like

renovate/lib/modules/manager/github-actions/extract.ts

Lines 195 to 216 in 71bb1e2

	function detectDatasource(registryUrl: string): PackageDependency {
	const platform = detectPlatform(registryUrl);
	switch (platform) {
	case 'forgejo':
	return {
	registryUrls: [registryUrl],
	datasource: ForgejoTagsDatasource.id,
	};
	case 'gitea':
	return {
	registryUrls: [registryUrl],
	datasource: GiteaTagsDatasource.id,
	};
	case 'github':
	return { registryUrls: [registryUrl] };
	}
	return {
	skipReason: 'unsupported-url',
	};
	}

[pbaylas]

I've run into the same issue with a pre-commit repo from code.forgejo.org:

  - repo: https://code.forgejo.org/forgejo/runner
    rev: v12.13.0
    hooks:
      - id: forgejo-runner-validate

[wwuck]

@viceice I tried this diff

diff --git a/lib/modules/manager/pre-commit/extract.ts b/lib/modules/manager/pre-commit/extract.ts
index 91b744c21..2b1fb0ff0 100644
--- a/lib/modules/manager/pre-commit/extract.ts
+++ b/lib/modules/manager/pre-commit/extract.ts
@@ -9,6 +9,8 @@ import { detectPlatform } from '../../../util/common.ts';
 import { find } from '../../../util/host-rules.ts';
 import { newlineRegex, regEx } from '../../../util/regex.ts';
 import { parseSingleYaml } from '../../../util/yaml.ts';
+import { ForgejoTagsDatasource } from '../../datasource/forgejo-tags/index.ts';
+import { GiteaTagsDatasource } from '../../datasource/gitea-tags/index.ts';
 import { GithubTagsDatasource } from '../../datasource/github-tags/index.ts';
 import { GitlabTagsDatasource } from '../../datasource/gitlab-tags/index.ts';
 import { parseLine } from '../gomod/line-parser.ts';
@@ -35,7 +37,29 @@ function determineDatasource(
   repository: string,
   hostname: string,
 ): { datasource?: string; registryUrls?: string[]; skipReason?: SkipReason } {
-  if (hostname === 'github.com' || detectPlatform(repository) === 'github') {
+  const platform = detectPlatform(repository);
+
+  if (platform === 'forgejo') {
+    logger.debug(
+      { repository, hostname },
+      'Found forgejo dependency with custom registryUrl',
+    );
+    return {
+      datasource: ForgejoTagsDatasource.id,
+      registryUrls: [`https://${hostname}`],
+    };
+  }
+  if (platform === 'gitea') {
+    logger.debug(
+      { repository, hostname },
+      'Found gitea dependency with custom registryUrl',
+    );
+    return {
+      datasource: GiteaTagsDatasource.id,
+      registryUrls: [`https://${hostname}`],
+    };
+  }
+  if (hostname === 'github.com' || platform === 'github') {
     logger.debug({ repository, hostname }, 'Found github dependency');
     return { datasource: GithubTagsDatasource.id };
   }
@@ -43,7 +67,7 @@ function determineDatasource(
     logger.debug({ repository, hostname }, 'Found gitlab dependency');
     return { datasource: GitlabTagsDatasource.id };
   }
-  if (detectPlatform(repository) === 'gitlab') {
+  if (platform === 'gitlab') {
     logger.debug(
       { repository, hostname },
       'Found gitlab dependency with custom registryUrl',
@@ -64,6 +88,8 @@ function determineDatasource(
     return { skipReason: 'unknown-registry', registryUrls: [hostname] };
   }
   for (const [hostType, sourceId] of [
+    ['forgejo', ForgejoTagsDatasource.id],
+    ['gitea', GiteaTagsDatasource.id],
     ['github', GithubTagsDatasource.id],
     ['gitlab', GitlabTagsDatasource.id],
   ]) {
@@ -77,7 +103,7 @@ function determineDatasource(
   }
   logger.debug(
     { repository, registry: hostUrl },
-    'Provided hostname did not match any of the hostRules of hostType github nor gitlab',
+    'Provided hostname did not match any of the hostRules of hostType forgejo, gitea, github nor gitlab',
   );
   return { skipReason: 'unknown-registry', registryUrls: [hostname] };
 }

but it's not working yet.

DEBUG: packageFiles with updates (repository=wwuck/renovate-pre-commit-codeberg, baseBranch=main)
       "config": {
         "pre-commit": [
           {
             "deps": [
               {
                 "datasource": "forgejo-tags",
                 "registryUrls": ["https://codeberg.org"],
                 "depName": "gherynos/pre-commit-java",
                 "depType": "repository",
                 "packageName": "gherynos/pre-commit-java",
                 "currentValue": "v0.6.37",
                 "updates": [],
                 "versioning": "semver-coerced",
                 "warnings": [
                   {
                     "topic": "gherynos/pre-commit-java",
                     "message": "Failed to look up forgejo-tags package gherynos/pre-commit-java: no-result"
                   }
                 ]
               }
             ],
             "packageFile": ".pre-commit-config.yaml"
           }
         ]
       }

Any suggestions on what I have missed? I can submit a PR when I have it working.

[viceice]

I don't know, you need to check the logs or start debugging

[wwuck]

Hmm so I tried debugging and keep getting ETIMEDOUT every time in https://github.com/renovatebot/renovate/blob/main/lib/util/http/got.ts#L45. The request timeout is clearly set to 60000 so I'm surprised this error is being thrown.

DEBUG: hostRules: no authentication for codeberg.org (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Using queue: host=codeberg.org, concurrency=16 (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: GET https://codeberg.org/api/v1/repos/gherynos/pre-commit-java/tags = (code=ETIMEDOUT, statusCode=-1 retryCount=2, duration=251) (repository=wwuck/renovate-pre-commit-codeberg)
DEBUG: Datasource connection error (repository=wwuck/renovate-pre-commit-codeberg)
       "datasource": "forgejo-tags",
       "packageName": "gherynos/pre-commit-java",
       "url": undefined,
       "errCode": "ETIMEDOUT"

{
  "name": "RequestError",
  "code": "ETIMEDOUT",
  "timings": {
    "start": 1784526438893,
    "socket": 1784526438893,
    "lookup": 1784526438894,
    "error": 1784526439145,
    "phases": {
      "wait": 0,
      "dns": 1,
      "total": 252
    }
  },
  "options": {
    "agent": {},
    "decompress": true,
    "timeout": {
      "request": 60000
    },
    "prefixUrl": "",
    "ignoreInvalidCookies": false,
    "context": {
      "hostType": "forgejo-tags"
    },
    "hooks": {
      "init": [],
      "beforeRequest": [],
      "beforeError": [],
      "beforeRedirect": [],
      "beforeRetry": [],
      "beforeCache": [],
      "afterResponse": []
    },
    "followRedirect": true,
    "maxRedirects": 10,
    "throwHttpErrors": true,
    "username": "",
    "password": "",
    "http2": false,
    "allowGetBody": false,
    "copyPipedHeaders": true,
    "headers": {
      "user-agent": "Renovate/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
      "accept": "application/json",
      "accept-encoding": "gzip, deflate, br, zstd"
    },
    "methodRewriting": false,
    "retry": {
      "limit": 2,
      "methods": [
        "GET",
        "PUT",
        "HEAD",
        "DELETE",
        "OPTIONS",
        "TRACE"
      ],
      "statusCodes": [
        408,
        413,
        429,
        500,
        502,
        503,
        504,
        521,
        522,
        524
      ],
      "errorCodes": [
        "ETIMEDOUT",
        "ECONNRESET",
        "EADDRINUSE",
        "ECONNREFUSED",
        "EPIPE",
        "ENOTFOUND",
        "ENETUNREACH",
        "EAI_AGAIN"
      ],
      "maxRetryAfter": 0,
      "backoffLimit": null,
      "noise": 100,
      "enforceRetryRules": false
    },
    "method": "GET",
    "cacheOptions": {},
    "https": {},
    "resolveBodyOnly": false,
    "isStream": false,
    "responseType": "json",
    "url": "https://codeberg.org/api/v1/repos/gherynos/pre-commit-java/tags",
    "pagination": {
      "countLimit": null,
      "backoff": 0,
      "requestLimit": 10000,
      "stackAllItems": false
    },
    "setHost": true,
    "enableUnixSockets": false,
    "strictContentLength": false
  }
}

I have read somewhere that Codeberg doesn't apply browser checks like anubis to the api endpoints, so I doubt that is affecting it.

I'll keep trying to see if anything else comes up here.

[wwuck]

I tested my change with both the codeberg.org and code.forgejo.org pre-commit hooks in .pre-commit-config.yaml but they both get an ETIMEDOUT error so I can confirm it's not specific to codeberg.org.

EDIT: so I ran again with the nodejs debugger and this time the code.forgejo.org hook gets an update by renovate but the codeberg.org hook gets ETIMEDOUT 😕