Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Configurability host rules for datasources and changelogs #13654

Closed

Conversation

rv2673
Copy link
Contributor

@rv2673 rv2673 commented Jan 19, 2022

Changes:

  • Make url encoding consistent gitlab-changelog consistent with other gitlab datasources.
  • Add hostType for gitlab changelog lookups, to differentiate them from platform calls.
  • Add hostType for github changelog lookups, to differentiate them from platform calls.
  • Add fallback mechanism for bitbucket datasource hostTypes(like is done for gitlab and github)
  • Add missing hostType to bitbucket-tags datasource.
  • Add missing hostType to gitlab-packages datasource. (Was already used in fallback configuration)
  • Extend tests for hostRules fallback mechanism
  • Extend regression tests for fallback constants.

Context:

Applying hostRules for specific datsources is usually done by adding hostType to be matched to the rule, containing the datasource id. However this is not possible for all datasources, since a few don't add their id as hostType, but instead use the default hostType added by the platform specific Http client, which usually matches the platformId. Which doesn't make it possible to differentiate between datasource and platform calls. They also cannot be overriden when running on that platform as noticed in ##13497

This is the case for the following datasources:

  • gitlab-packages
  • bitbucket-tags

The changelog modules which run after datasource lookups also don't add a hostType make it impossible to differentiate them between platform calls and lookups for changelogs. So one can configure token for gitlab-tags, but fail on changelog lookup.(See #13497)

This PR resolves this by treating the changelogs modules like any other (platform) datasource module(gitlab-tags, github-tags), which add a specific hostType to make it possible to add specific hostRules, while still having a fallback to the platform rules in case no specific rules are present.
The added hostTypes are

  • gitlab-changelog
  • github-changelog

Closes #13497
Replaces #13577

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to split this into smaller pieces so that we don't need to roll back the the whole thing if there's a problem. Seems like we can have these separate PRs:

  • BitBucket tags/http alignment with github/gitlab
  • Normalize %2F for GitLab
  • *-changelog new hostType idea

@rv2673
Copy link
Contributor Author

rv2673 commented Jan 20, 2022

@rarkins Have done as you have requested(was quite some git work to split the test commits)
Have seperated it in the following ones(I also seperated out the gitlab-packages change, in line with the reasoning of the smaller requests)

Resulting PR's:

@rv2673 rv2673 closed this Jan 20, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 20, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Differing behaviour retrieving gitlab-tags and releases between datasource and changelog
2 participants