GitLab CI external repo #1238

philipqnguyen · 2020-06-25T06:33:15Z

Resolves #1044

It is common for code repos to be hosted on Github while using
Gitlab only as a CI/CD service.

Info on using Gitlab only as CI/CD service for external repos:
https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/

Current limitations:

Unfortunately, Gitlab currently does not expose the external repo's
url, so users will have to manually set the repo's URL as an ENV var
DANGER_PROJECT_REPO_URL
(for ex., DANGER_PROJECT_REPO_URL=https://github.com/procore/blueprinter).
Additionally, RequestSources::Github#validates_as_ci? is tricky now because
the git remote -v returns origin as gitlab.com rather than github.com. The
reasoning is that even though the actual repo is on Github.com, GitlabCI maintains a
mirror on Gitlab.com and the CI checkout the code from the Gitlab.com mirror.

philipqnguyen · 2020-06-25T06:36:54Z

lib/danger/ci_source/gitlab_ci.rb

+
+    def slug_from(env)
+      if env["DANGER_PROJECT_REPO_URL"]
+        env["DANGER_PROJECT_REPO_URL"].split('/').last(2).join('/')


gitlab doesn't seem to give us any env var that points to the external repo's URL. It only points to gitlab's mirror URL. Users will need to set this ENV var themselves

philipqnguyen · 2020-06-25T06:43:01Z

lib/danger/request_sources/github/github.rb

@@ -40,6 +40,10 @@ def get_pr_from_branch(repo_name, branch_name, owner)
        end
      end

+      def validates_as_ci?
+        true
+      end


In the base class, it is looking up the git remote origin to see if it contains github.com:

danger/lib/danger/request_sources/request_source.rb

Lines 39 to 42 in 61a09f8

# @return [Boolean] whether scm.origins is a valid git repository or not

def validates_as_ci?

!!self.scm.origins.match(%r{#{Regexp.escape self.host}(:|/)(.+/.+?)(?:\.git)?$})

end

Unfortunately, RequestSources::Github#validates_as_ci? is tricky now because
the git remote -v returns origin gitlab.com rather than github.com. The
reasoning is that even though the actual repo is on Github.com, GitlabCI maintains a
mirror on Gitlab.com and the CI checkout the code from there.

I could change this to also accept an origin that matches gitlab.com if the ci_source is Gitlab, however I am not sure I understand why there is a need to validate this at all, especially if we are already performing the validates_as_api_source?.

I think this is probably reasonable

philipqnguyen · 2020-06-25T06:52:49Z

lib/danger/ci_source/gitlab_ci.rb

-      @repo_slug = env["CI_MERGE_REQUEST_PROJECT_PATH"] || env["CI_PROJECT_PATH"]
-      @project_url = env["CI_MERGE_REQUEST_PROJECT_URL"] || env["CI_PROJECT_URL"]
+      @repo_slug = slug_from(env)
+      @project_url = env["DANGER_PROJECT_REPO_URL"] || env["CI_MERGE_REQUEST_PROJECT_URL"] || env["CI_PROJECT_URL"]


On an unrelated note, there is a separate PR #1237 to remove this project_url as it does not seem to be used anymore.

philipqnguyen · 2020-07-06T16:26:15Z

I applied your suggestions @yg
Please let me know if anything else requires changing.

philipqnguyen · 2020-07-09T16:51:58Z

@orta What do you think of this and are there any changes you want to see?

orta

The validates_as_ci is the bit that really throws me off here, I'm gonna accept because it does look like I'm never going to take the time to dive into that properly at this point.

orta · 2020-07-09T16:57:51Z

lib/danger/request_sources/github/github.rb

@@ -40,6 +40,10 @@ def get_pr_from_branch(repo_name, branch_name, owner)
        end
      end

+      def validates_as_ci?
+        true
+      end


I think this is probably reasonable

orta · 2020-07-09T16:58:57Z

If you rebase I'll merge 👍

It is common for code repos to be hosted on Github while using Gitlab only as a CI/CD service. Info on using Gitlab only as CI/CD service for external repos: https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/ Current limitations: Unfortunately, Gitlab currently does not expose the external repo's url, so we have to set it in an ENV var as DANGER_PROJECT_REPO_URL.

The previous specs assumes all the PRs (or merge requests) are made on a Gitlab hosted code repository. We move those specs to its own context.

The following tests are for methods that will have an alternate value if the code repo is hosted on Github.

GitlabCI with a Github hosted repository will actually have its git remote origin point to gitlab.com and not to github.com. The reason is that GitlabCI maintains a mirror of the Github Repo and that the git remote in the CI points to the mirror rather than to the actual git repository on Github. I'm not sure if I see a need for maintain this #validates_as_ci? for RequestSources...

Brand typo Co-authored-by: Yogi <me@yogi.codes>

danger-public · 2020-07-09T18:54:33Z

	1 Error
🚫	Tests have failed, see below for more information.

Tests:

File	Name
gitlab_spec.rb	Danger::RequestSources::GitLab valid server response #update_pull_request! supports inline comments edits existing inline comment instead of creating a new one if file/line matches
gitlab_spec.rb	Danger::RequestSources::GitLab valid server response #update_pull_request! supports inline comments deletes non-sticky comments if no violations are present
gitlab_spec.rb	Danger::RequestSources::GitLab valid server response #update_pull_request! supports inline comments deletes inline comments if those are no longer relevant
gitlab_spec.rb	Danger::RequestSources::GitLab valid server response #update_pull_request! supports inline comments skips inline comments for files that are not part of the diff

Generated by 🚫 Danger

philipqnguyen · 2020-07-09T19:26:26Z

@orta, I rebased the PR, but tests are failing because a new gitlab gem was released on July 6th, v4.16.0, and it contains a bug: NARKOZ/gitlab#574

I believe a solution would be to specify gitlab < 4.16.0

orta · 2020-07-09T19:32:22Z

Nah, not need to lock. I don't want to be accidentally missing something in the future by locking the version. I'm OK with CI being red until it's figured out, there's enough greens to validate that you didn't break something.

NARKOZ · 2020-07-12T21:27:54Z

@orta, I rebased the PR, but tests are failing because a new gitlab gem was released on July 6th, v4.16.0, and it contains a bug: NARKOZ/gitlab#574

I believe a solution would be to specify gitlab < 4.16.0

I've released version 4.16.1 with a fix.

orta · 2020-07-13T12:02:40Z

<3

philipqnguyen commented Jun 25, 2020

View reviewed changes

philipqnguyen mentioned this pull request Jun 25, 2020

Github + Gitlab CI, get_repo_source is getting gitlab (it's a mirror), but the code is on github #1044

Closed

orta approved these changes Jul 9, 2020

View reviewed changes

philipqnguyen and others added 6 commits July 9, 2020 11:43

Move existing specs to gitlab hosted context

aaf6f15

The previous specs assumes all the PRs (or merge requests) are made on a Gitlab hosted code repository. We move those specs to its own context.

Add context for when PR made from Github repo

803c24c

The following tests are for methods that will have an alternate value if the code repo is hosted on Github.

Update changelog w/ Gitlab external CI/CD

5b082d7

Apply suggestions from code review

a16b209

Brand typo Co-authored-by: Yogi <me@yogi.codes>

philipqnguyen force-pushed the gitlab-ci-external-repo branch from 1b75fd1 to a16b209 Compare July 9, 2020 18:53

orta merged commit 8067dd5 into danger:master Jul 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitLab CI external repo #1238

GitLab CI external repo #1238

philipqnguyen commented Jun 25, 2020 •

edited

Loading

philipqnguyen Jun 25, 2020

philipqnguyen Jun 25, 2020

orta Jul 9, 2020

philipqnguyen Jun 25, 2020

philipqnguyen commented Jul 6, 2020

philipqnguyen commented Jul 9, 2020

orta left a comment

orta Jul 9, 2020

orta commented Jul 9, 2020

danger-public commented Jul 9, 2020

philipqnguyen commented Jul 9, 2020 •

edited

Loading

orta commented Jul 9, 2020

NARKOZ commented Jul 12, 2020

orta commented Jul 13, 2020

	# @return [Boolean] whether scm.origins is a valid git repository or not
	def validates_as_ci?
	!!self.scm.origins.match(%r{#{Regexp.escape self.host}(:\|/)(.+/.+?)(?:\.git)?$})
	end

GitLab CI external repo #1238

GitLab CI external repo #1238

Conversation

philipqnguyen commented Jun 25, 2020 • edited Loading

philipqnguyen Jun 25, 2020

Choose a reason for hiding this comment

philipqnguyen Jun 25, 2020

Choose a reason for hiding this comment

orta Jul 9, 2020

Choose a reason for hiding this comment

philipqnguyen Jun 25, 2020

Choose a reason for hiding this comment

philipqnguyen commented Jul 6, 2020

philipqnguyen commented Jul 9, 2020

orta left a comment

Choose a reason for hiding this comment

orta Jul 9, 2020

Choose a reason for hiding this comment

orta commented Jul 9, 2020

danger-public commented Jul 9, 2020

Tests:

philipqnguyen commented Jul 9, 2020 • edited Loading

orta commented Jul 9, 2020

NARKOZ commented Jul 12, 2020

orta commented Jul 13, 2020

philipqnguyen commented Jun 25, 2020 •

edited

Loading

philipqnguyen commented Jul 9, 2020 •

edited

Loading