Manage Gitlab certificate (#1249)

* Manage gitlab certificate * Build * Manage GITLAB_CUSTOM_CERTIFICATE quick build push * Fix python issue quick build push * Error msg doc * [MegaLinter] Apply linters fixes * Update config json schema quick build push Co-authored-by: nvuillam <nvuillam@users.noreply.github.com>
oxsecurity · Feb 6, 2022 · 1e261f6 · 1e261f6
1 parent 54dc5a5
commit 1e261f6
Show file tree

Hide file tree

Showing 20 changed files with 84 additions and 13 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,8 +6,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ## [Unreleased] (beta, main branch content)
 
+- Fixes
+  - Manage to use local certificate with Gitlab comments reporter using GITLAB_SSL_CERTIFICATE_PATH ([#1239](https://github.com/megalinter/megalinter/issues/1239))
+
 Note: Can be used with `megalinter/megalinter@beta` in your GitHub Action mega-linter.yml file, or with `megalinter/megalinter:beta` docker image
 
+- Fixes
+  - Gitlab Comments Reporter: allow to use certificates with variable GITLAB_CUSTOM_CERTIFICATE (or GITLAB_CERTIFICATE_PATH only if [PRE_COMMANDS](https://megalinter.github.io/configuration/#pre-commands) are used) ([#1239](https://github.com/megalinter/megalinter/issues/1239))
+
 - Doc
   - Update images with screen records gifs
   - Add publish artifact task in azure pipelines doc

diff --git a/Dockerfile b/Dockerfile
@@ -48,6 +48,7 @@ ARG PSSA_VERSION='latest'
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/docs/reporters/GitlabCommentReporter.md b/docs/reporters/GitlabCommentReporter.md
@@ -16,9 +16,11 @@ Click on hyperlinks to access detailed logs (click on **Download** in **Artifact
 
 ![config-gitlab-access-token](https://user-images.githubusercontent.com/17500430/151674446-1bcb1420-d9aa-4ae1-aaae-dcf51afb36ab.gif)
 
-| Variable                       | Description                                                                               | Default value            |
-|--------------------------------|-------------------------------------------------------------------------------------------|--------------------------|
-| GITLAB_COMMENT_REPORTER        | Activates/deactivates reporter                                                            | true                     |
-| GITLAB_API_URL                 | URL where the github API can be reached<br/>May be overridden if using self-hosted Gitlab | `https://api.gitlab.com` |
-| GITLAB_SERVER_URL              | URL of the Gitlab instance<br/>May be overridden if using self-hosted Gitlab              | `https://gitlab.com`     |
-| GITLAB_ACCESS_TOKEN_MEGALINTER | Must contain a Gitlab private access token defined with api access                        | <!-- -->                 |
+| Variable                       | Description                                                                                            | Default value            |
+|--------------------------------|--------------------------------------------------------------------------------------------------------|--------------------------|
+| GITLAB_COMMENT_REPORTER        | Activates/deactivates reporter                                                                         | true                     |
+| GITLAB_API_URL                 | URL where the github API can be reached<br/>May be overridden if using self-hosted Gitlab              | `https://api.gitlab.com` |
+| GITLAB_SERVER_URL              | URL of the Gitlab instance<br/>May be overridden if using self-hosted Gitlab                           | `https://gitlab.com`     |
+| GITLAB_ACCESS_TOKEN_MEGALINTER | Must contain a Gitlab private access token defined with api access                                     | <!-- -->                 |
+| GITLAB_CUSTOM_CERTIFICATE      | SSL certificate value to connect to Gitlab                                                             | <!-- -->                 |
+| GITLAB_CERTIFICATE_PATH        | Path to SSL certificate to connect to Gitlab (if SSL cert has been manually defined with PRE_COMMANDS) | <!-- -->                 |
diff --git a/flavors/ci_light/Dockerfile b/flavors/ci_light/Dockerfile
@@ -33,6 +33,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/dart/Dockerfile b/flavors/dart/Dockerfile
@@ -36,6 +36,7 @@ ARG GLIBC_VERSION='2.31-r0'
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/documentation/Dockerfile b/flavors/documentation/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/dotnet/Dockerfile b/flavors/dotnet/Dockerfile
@@ -40,6 +40,7 @@ ARG PSSA_VERSION='latest'
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/go/Dockerfile b/flavors/go/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/java/Dockerfile b/flavors/java/Dockerfile
@@ -36,6 +36,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/javascript/Dockerfile b/flavors/javascript/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/php/Dockerfile b/flavors/php/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/python/Dockerfile b/flavors/python/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/ruby/Dockerfile b/flavors/ruby/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/rust/Dockerfile b/flavors/rust/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/salesforce/Dockerfile b/flavors/salesforce/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/scala/Dockerfile b/flavors/scala/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/swift/Dockerfile b/flavors/swift/Dockerfile
@@ -35,6 +35,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/flavors/terraform/Dockerfile b/flavors/terraform/Dockerfile
@@ -39,6 +39,7 @@ FROM python:3.9.7-alpine3.13
 # APK Packages used by mega-linter core architecture
 RUN apk add --update --no-cache \
     bash \
+    ca-certificates \
     coreutils \
     curl \
     file \

diff --git a/megalinter/descriptors/schemas/megalinter-configuration.jsonschema.json b/megalinter/descriptors/schemas/megalinter-configuration.jsonschema.json
@@ -3314,13 +3314,34 @@
       "title": "GitHub Workspace",
       "type": "string"
     },
+    "GITLAB_ACCESS_TOKEN_MEGALINTER": {
+      "$id": "#/properties/GITLAB_ACCESS_TOKEN_MEGALINTER",
+      "default": true,
+      "description": "Gitlab access token with API access (can be user or project Access token)",
+      "title": "Gitlab Access Token",
+      "type": "string"
+    },
+    "GITLAB_CERTIFICATE_PATH": {
+      "$id": "#/properties/GITLAB_CERTIFICATE_PATH",
+      "default": true,
+      "description": "Path to SSL certificate if necessary",
+      "title": "Gitlab SSL Certificate path",
+      "type": "string"
+    },
     "GITLAB_COMMENT_REPORTER": {
       "$id": "#/properties/GITLAB_COMMENT_REPORTER",
       "default": true,
       "description": "Posts Mega-Linter results summary in the comments of the related merge request (if existing)",
       "title": "Activate Gitlab MR Comments reporter",
       "type": "boolean"
     },
+    "GITLAB_CUSTOM_CERTIFICATE": {
+      "$id": "#/properties/GITLAB_CUSTOM_CERTIFICATE",
+      "default": true,
+      "description": "Store your SSL certificate string value in a CI env variable named GITLAB_CUSTOM_CERTIFICATE if you want it to be copied by MegaLinter in a local certificate file that will be used by python gitlab lib",
+      "title": "Gitlab SSL Certificate value",
+      "type": "string"
+    },
     "GIT_FILTER_REGEX_EXCLUDE": {
       "$id": "#/properties/GIT_FILTER_REGEX_EXCLUDE",
       "title": "Excluding regex filter for GIT descriptor",

diff --git a/megalinter/reporters/GitlabCommentReporter.py b/megalinter/reporters/GitlabCommentReporter.py
@@ -7,6 +7,7 @@
 
 import gitlab
 from megalinter import Reporter, config
+from megalinter.pre_post_factory import run_command
 from megalinter.utils_reporter import build_markdown_summary
 
 
@@ -47,16 +48,39 @@ def produce_report(self):
             action_run_url = config.get("CI_JOB_URL", "")
             p_r_msg = build_markdown_summary(self, action_run_url)
 
-            # Post comment on merge request if found
+            # Build gitlab options
+            gitlab_options = {}
+            # auth token
             if config.get("GITLAB_ACCESS_TOKEN_MEGALINTER", "") != "":
-                gl = gitlab.Gitlab(
-                    gitlab_server_url,
-                    private_token=config.get("GITLAB_ACCESS_TOKEN_MEGALINTER"),
+                gitlab_options["private_token"] = config.get(
+                    "GITLAB_ACCESS_TOKEN_MEGALINTER"
                 )
             else:
-                gl = gitlab.Gitlab(
-                    gitlab_server_url, job_token=config.get("CI_JOB_TOKEN")
+                gitlab_options["job_token"] = config.get("CI_JOB_TOKEN")
+            # Certificate management
+            gitlab_certificate_path = config.get("GITLAB_CERTIFICATE_PATH", "")
+            if config.get("GITLAB_CUSTOM_CERTIFICATE", "") != "":
+                # Certificate value defined in an ENV variable
+                cert_value = config.get("GITLAB_CUSTOM_CERTIFICATE")
+                gitlab_certificate_path = "/etc/ssl/certs/gitlab-cert.crt"
+                with open(gitlab_certificate_path, "w", encoding="utf-8") as cert_file:
+                    cert_file.write(cert_value)
+                    logging.debug(
+                        f"Updated {gitlab_certificate_path} with certificate value {cert_value}"
+                    )
+            if gitlab_certificate_path != "":
+                # Update certificates and set cert path in gitlab options
+                run_command(
+                    {"cwd": "root", "command": "update-ca-certificates"},
+                    "GitlabCommentReporter",
+                    self.master,
                 )
+                gitlab_options["ssl_verify"] = gitlab_certificate_path
+            # Create gitlab connection
+            logging.debug(
+                f"[GitlabCommentReporter] Logging to {gitlab_server_url} with {str(gitlab_options)}"
+            )
+            gl = gitlab.Gitlab(gitlab_server_url, **gitlab_options)
             # Get gitlab project
             try:
                 project = gl.projects.get(gitlab_project_id)
@@ -143,5 +167,7 @@ def display_auth_error(self, e):
         logging.error(
             "[Gitlab Comment Reporter] You may need to define a masked Gitlab CI/CD variable "
             "MEGALINTER_ACCESS_TOKEN containing a personal token with scope 'api'\n"
-            "(if already defined, your token is probably invalid)" + str(e)
+            "(if already defined, your token is probably invalid)\n"
+            "If you are using local certificate, you also may need to define variables "
+            "GITLAB_CUSTOM_CERTIFICATE or GITLAB_CERTIFICATE_PATH" + str(e)
         )