feat(repo/changelogs): allow user configuration of source url

[Gabriel-Ladzaretti]

Changes

Introduces a new config option named overwriteSourceUrl which is valid within the PackageRules object only (not enforced, a warning message will be logged).

When set, this url will be used as the source of changelogs retrieval.
This url is now passed a function parameter to both getChangeLogJSON functions.

Cache namespace changelog-github/github-release which is used for caching the Compare Source links. which is originally structured as - manager:depname:prevVer:nextVer ,will now also contain the repository/pathname to prevent conflicts + use datasource instead of manager as part of the key., is now sourceUrl:depname:prevVer:nextVer.

Note: sourceUrl is restored to its original value after changelog retrieval, this is due the fact that it is part of the repository cache. and if im not mistaken changelogs has no meaning in the context of repository cache but the original datasource fetched sourceUrl does.

Context

closes #16011

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[rarkins]

Shouldn't the cache namespace be related to datasource and not manager?

[Gabriel-Ladzaretti]

Shouldn't the cache namespace be related to datasource and not manager?

probably, as the source url is being set in the datasource stage.
should i change it in this PR?

[rarkins]

Yes I think if you're making changes then now is a good time. Let's consider if other fields need changing too

[Gabriel-Ladzaretti]

Yes I think if you're making changes then now is a good time. Let's consider if other fields need changing too

I think this might be deeper than just changing to datasource as it's type is string | undefiend
while manager is strictly a string as per

• refactor: better manager types #15708

I mean we can asset not null (! operator) but im not sure its entirely true at runtime

edit: on second thought, this might not be a problem

renovate/lib/workers/repository/update/pr/changelog/source-github.ts

Lines 36 to 42 in e4dbd4a

	const versioning = config.versioning!;
	const currentVersion = config.currentVersion!;
	const newVersion = config.newVersion!;
	const sourceUrl = config.sourceUrl!;
	const sourceDirectory = config.sourceDirectory!;
	const depName = config.depName!;
	const manager = config.manager;

[Gabriel-Ladzaretti]

other namespaces and cache keys

GitHub

mem cache

renovate/lib/workers/repository/update/pr/changelog/source-github.ts

Line 22 in e4dbd4a

const cacheKey = `getTags-${endpoint}-${repository}`;

package cache - key modified: repository added, manager change to datasource

renovate/lib/workers/repository/update/pr/changelog/source-github.ts

Lines 122 to 125 in e4dbd4a

	const cacheNamespace = 'changelog-github-release';
	function getCacheKey(prev: string, next: string): string {
	return `${manager}:${depName}:${prev}:${next}`;
	}

GitLab

memCache

renovate/lib/workers/repository/update/pr/changelog/source-gitlab.ts

Line 22 in e4dbd4a

const cacheKey = `getTags-${endpoint}-${versionScheme}-${repository}`;

package cache - key modified: pathname added, manager change to datasource

renovate/lib/workers/repository/update/pr/changelog/source-gitlab.ts

Line 15 in e4dbd4a

const cacheNamespace = 'changelog-gitlab-release';

renovate/lib/workers/repository/update/pr/changelog/source-gitlab.ts

Lines 97 to 99 in e4dbd4a

	function getCacheKey(prev: string, next: string): string {
	return `${manager}:${depName}:${prev}:${next}`;
	}

Release notes

packageCache

renovate/lib/workers/repository/update/pr/changelog/release-notes.ts

Lines 394 to 399 in e4dbd4a

	const cacheNamespace = `changelog-${input.project.type}-notes@v2`;
	function getCacheKey(version: string): string {
	return `${repository}:${
	sourceDirectory ? `${sourceDirectory}:` : ''
	}${version}`;
	}

mem cache

renovate/lib/workers/repository/update/pr/changelog/release-notes.ts

Lines 60 to 62 in e4dbd4a

	const cacheKey = `getReleaseList-${project.apiBaseUrl!}-${
	project.repository
	}`;

renovate/lib/workers/repository/update/pr/changelog/release-notes.ts

Lines 271 to 273 in e4dbd4a

	const cacheKey = `getReleaseNotesMdFile@v2-${project.repository}${
	project.sourceDirectory ? `-${project.sourceDirectory}` : ''
	}-${project.apiBaseUrl!}`;

package cache

renovate/lib/workers/repository/update/pr/changelog/release-notes.ts

Lines 394 to 399 in e4dbd4a

	const cacheNamespace = `changelog-${input.project.type}-notes@v2`;
	function getCacheKey(version: string): string {
	return `${repository}:${
	sourceDirectory ? `${sourceDirectory}:` : ''
	}${version}`;
	}

[Gabriel-Ladzaretti]

now using datasource as part of the relevant cache keys.
commit a760e03

[viceice]

if it's only used for changelogs, shouldn't we name it customChangelogUrl ? and use that if set in changelogs?

[Gabriel-Ladzaretti]

if it's only used for changelogs, shouldn't we name it customChangelogUrl ? and use that if set in changelogs?

you mean to change all sourceUrl -> customChangelogUrl ?? sourceUrl!, so basically customChangelogUrl have precedence over sourceUrl

i guess it depends on how many places we are using sourceUrl directly,

[viceice]

yes, but only in changelogs.

[viceice]

maybe also needs customChangelogsDirectory

[viceice]

those should then passt to all functions in changelogs, so they are evaluated once, like platform detection

[Gabriel-Ladzaretti]

those should then passt to all functions in changelogs, so they are evaluated once, like platform detection

im not that familiar with this flow, do you mean directory in terms of caching? if so, in current state all changelogs even overridden ones are getting cached, the keying was changed in a defensive way such that if a user overwrites the sourceUrl of a well known dep its key will reflect that.

thats why in the POC i used the same dependency, one repo uses a custom url and the next uses the default one and there was no interference between them

[rarkins]

maybe also needs customChangelogsDirectory

It would be nice if we can overcome the barriers to reusing all our existing field names, rather than defining each one again

[viceice]

see other comments

[renovate-release]

🎉 This PR is included in version 32.172.0 🎉

The release is available on:

• GitHub release
• 32.172.0

Your semantic-release bot 📦🚀