Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(gitlab): Consistent urlencoding and call source specific hostType for gitlab (changelogs, packages, tags, releases) #13577

Closed
wants to merge 3 commits into from
Closed

fix(gitlab): Consistent urlencoding and call source specific hostType for gitlab (changelogs, packages, tags, releases) #13577

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
50737b9
refactor(gitlab-changelog): Make url encoding consistent
rv2673 Jan 16, 2022
7f06d06
fix(gitlab):Make it possible to differentiate calls for packages and …
rv2673 Jan 16, 2022
d25419b
Merge branch 'main' into fix-13497
rarkins Jan 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/constants/platforms.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ export const GITLAB_API_USING_HOST_TYPES = [
'gitlab-releases',
'gitlab-tags',
'gitlab-packages',
'gitlab-changelog',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@viceice are you sure you like this idea?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know a better solution yet. 🤷‍♂️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have looked for alternative solutions to solve the problem.

The problem is that it is currently impossible to apply hostRules to calls made by the changelog modules. This due to the fact that they use the platform identifier as hostType, which cannot have any rules applied to them since the automatically generated host rule for the platform is always applied last, overriding the authentication set by any previous rule.

Most datasources apply their datasource id to their Http client which adds it as hostType, making it possible to set hostRules for specific datasources. That is also what gitlab-tags and gitlab-releases datasources do. The hostRules also have a fallback mechanism for sources that use Gitlab or Github as source. This makes it possible for gitlab-tags and gitlab-releases to have specific rules configured for private repositories and otherwise fallback to the platform configuration. The changelog http clients,(which are called after datasource lookups) however always use the platform identifier as hostType, which as previously mentioned cannot be overridden.

I can think of only the 3 options to make hostRules configurable for changelogs so that they don't fail when datasources are configured with their own authentication which is not applied to subsequent changelog lookups.

  1. Moving automatically generated platform rule to the beginning, making it possible to override automatically generated rule for call using the platform identifier. Making it possible to override the configured renovate platform token. However there is no ways to differentiate between preset, platform and changelog calls.

    • Which might also break configurations where there are silently ignored hostRules for the platformId hostType.

  2. In the case of the Gitlab changelog retrieval, it does 3 types of calls, tag lookup, checking for existence of changelog md(tree lookup) and looking up releases.

    • tags lookup could use the same hostType that gitlab-tags datsource uses
    • release lookup could use the same hostType that gitlab-releases datsource uses
    • the tree api lookup call is done also done in the platform module with platform identifier as hostType, but that was the problem, so new identifier needs to be introduced, since both tags and releases types calls don't make sense.

  3. Treat the changelog module just like a gitlab-releases or gitlab-tags datasource with its own identifier used as hostType, making it possible to differentiate from other platform calls and apply rules to them, and adding it to the fallback list. (The solution above, which I personally like best, but I am biased.)

];
2 changes: 1 addition & 1 deletion lib/datasource/gitlab-packages/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ export class GitlabPackagesDatasource extends Datasource {

constructor() {
super(datasource);
this.http = new GitlabHttp();
this.http = new GitlabHttp(datasource);
}

static getGitlabPackageApiUrl(
Expand Down
54 changes: 27 additions & 27 deletions lib/workers/pr/changelog/__snapshots__/gitlab.spec.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tags?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tags?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -67,7 +67,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -78,7 +78,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -89,7 +89,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -100,7 +100,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -111,7 +111,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -122,7 +122,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -133,7 +133,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -144,7 +144,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
]
`;
Expand Down Expand Up @@ -315,7 +315,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tags?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tags?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -326,7 +326,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -337,7 +337,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -348,7 +348,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -359,7 +359,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -370,7 +370,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -381,7 +381,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -392,7 +392,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -403,7 +403,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
]
`;
Expand Down Expand Up @@ -464,7 +464,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tags?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tags?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -475,7 +475,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -486,7 +486,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -497,7 +497,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -508,7 +508,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -519,7 +519,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -530,7 +530,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -541,7 +541,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/repository/tree?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/repository/tree?per_page=100",
},
Object {
"headers": Object {
Expand All @@ -552,7 +552,7 @@ Array [
"user-agent": "RenovateBot/0.0.0-semantic-release (https://github.com/renovatebot/renovate)",
},
"method": "GET",
"url": "https://gitlab.com/api/v4/projects/meno%2fdropzone/releases?per_page=100",
"url": "https://gitlab.com/api/v4/projects/meno%2Fdropzone/releases?per_page=100",
},
]
`;
Expand Down
Loading