feat: new plugin jenkins-github-integ

[aFlyBird0]

Signed-off-by: Bird aflybird0@gmail.com

Pre-Checklist

Note: please complete ALL items in the following checklist.

• I have read through the CONTRIBUTING.md documentation.
• My code has the necessary comments and documentation (if needed).
• I have added relevant tests

Description

Related Issues

#920

New Behavior (screenshots if needed)

config

---
# core config
varFile: "" # If not empty, use the specified external variables config file
toolFile: "" # If not empty, use the specified external tools config file
state: # state config, backend can be local or s3
  backend: local
  options:
    stateFile: devstream-jenkins-all.state

---

tools:
  # name of the tool
  - name: jenkins
    # id of the tool instance
    instanceID: default
    # format: name.instanceID; If specified, dtm will make sure the dependency is applied first before handling this tool.
    dependsOn: [ ]
    # options for the plugin
    options:
      # if true, the plugin will use hostpath to create a pv named `jenkins-pv`
      # and you should create the volumes directory manually, see plugin doc for details.
      test_env: true
      # need to create the namespace or not, default: false
      create_namespace: true
      # Helm repo information
      repo:
        # name of the Helm repo
        name: jenkins
        # url of the Helm repo
        url: https://charts.jenkins.io
      # Helm chart information
      chart:
        # name of the chart
        chart_name: jenkins/jenkins
        # release name of the chart
        release_name: dev
        # k8s namespace where jenkins will be installed
        namespace: jenkins
        # whether to wait for the release to be deployed or not
        wait: true
        # the time to wait for any individual Kubernetes operation (like Jobs for hooks). This defaults to 5m0s
        timeout: 10m
        # whether to perform a CRD upgrade during installation
        upgradeCRDs: true
        # custom configuration. You can refer to [Jenkins values.yaml](https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/values.yaml)
        values_yaml: |
          persistence:
            # for prod env: the existent storageClass, please change it
            # for test env: just ignore it, but don't remove it
            storageClass: jenkins-pv
          serviceAccount:
            create: false
            name: jenkins
          controller:
            serviceType: NodePort
            nodePort: 32000
            additionalPlugins:
              # install "GitHub Pull Request Builder" plugin, see https://plugins.jenkins.io/ghprb/ for more details
              - ghprb
              # install "OWASP Markup Formatter" plugin, see https://plugins.jenkins.io/antisamy-markup-formatter/ for more details
              - antisamy-markup-formatter
            # Enable HTML parsing using OWASP Markup Formatter Plugin (antisamy-markup-formatter), useful with ghprb plugin.
            enableRawHtmlMarkupFormatter: true
            # Jenkins Configuraction as Code, refer to https://plugins.jenkins.io/configuration-as-code/ for more details
            # notice: All configuration files that are discovered MUST be supplementary. They cannot overwrite each other's configuration values. This creates a conflict and raises a ConfiguratorException.
            JCasC:
              defaultConfig: true
              # each key-value in configScripts will be added to the ${JENKINS_HOME}/casc_configs/ directory as a file.
              configScripts:
                # this will create a file named "safe_html.yaml" in the ${JENKINS_HOME}/casc_configs/ directory.
                # it is used to configure the "Safe HTML" plugin.
                # filename must meet RFC 1123, see https://tools.ietf.org/html/rfc1123 for more details
  - name: jenkins-pipeline-kubernetes
    # id of the tool instance
    instanceID: default
    # format: name.instanceID; If specified, dtm will make sure the dependency is applied first before handling this tool.
    dependsOn: [ "jenkins.default" ]
    # options for the plugin
    options:
      jenkins:
        # jenkinsUrl, format: hostname:port
        url: ${{jenkins.default.outputs.jenkinsURL}}
        # jenkins user name, default: admin
        user: admin
        # jenkins password, you have 2 options to set the password:
        # 1. use outputs of the `jenkins` plugin, see docs for more details
        # 2. set the `JENKINS_PASSWORD` environment variable
        # 3. fill in the password in this field(not recommended)
        # if all set, devstream will read the password from the config file first.
        password: ${{jenkins.default.outputs.jenkinsPasswordOfAdmin}}
        # jenkins job name, mandatory
        jobName: jenkins-plugin-test
        # path to the pipeline file, relative to the git repo root directory. default: Jenkinsfile
        pipelineScriptPath: Jenkinsfile
      # github repo url where the pipeline script is located. mandatory
      githubRepoUrl: https://github.com/aFlyBird0/jenkins-file-test.git
  - name: jenkins-github-integ
    # id of the tool instance
    instanceID: default
    # format: name.instanceID; If specified, dtm will make sure the dependency is applied first before handling this tool.
    dependsOn: [ "jenkins.default" ]
    # options for the plugin
    options:
      jenkins:
        # jenkinsUrl, format: hostname:port
        url: ${{jenkins.default.outputs.jenkinsURL}}
        # override the jenkins url to expose the jenkins to the GitHub webhook, can be empty.
        urlOverride: https://891e-125-111-206-162.ap.ngrok.io
        # jenkins user name, default: admin
        user: admin
        # jenkins password, you have 3 options to set the password:
        # 1. use outputs of the `jenkins` plugin, see docs for more details
        # 2. set the `JENKINS_PASSWORD` environment variable
        # 3. fill in the password in this field(not recommended)
        # if all set, devstream will read the password from the config file first.
        password: ${{jenkins.default.outputs.jenkinsPasswordOfAdmin}}
        # jenkins job name, mandatory
        jobName: job-pr
        # path to the pipeline file, relative to the git repo root directory. default: Jenkinsfile-pr
        pipelineScriptPath: Jenkinsfile-pr
      helm:
        # namespace of the jenkins, default: jenkins
        namespace: jenkins
        # release name of the jenkins helm chart, mandatory
        releaseName: dev
      # GitHub repo where to put the pipeline script and project. mandatory
      githubRepoUrl: https://github.com/aFlyBird0/jenkins-file-test
      adminList:
        - aFlyBird0

run

❯ kind delete clusters dev && cd ~/tmp/k8s && kind create cluster --config kind-cluster.yaml --image=kindest/node:v1.22.0 --name=dev
Deleted clusters: ["dev"]
Creating cluster "dev" ...
 ✓ Ensuring node image (kindest/node:v1.22.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-dev"
You can now use your cluster with:

kubectl cluster-info --context kind-dev

Thanks for using kind! 😊
❯ docker exec -it dev-control-plane sh -c "mkdir -p /Users/lhp/data/jenkins-volume/ && chown -R 1000:1000 /Users/lhp/data/jenkins-volume/ && exit"
❯ cd ~/Code/Go/opensource/devstream && rm devstream-jenkins-all.state && ./dtm apply -f jenkins-all.yaml -y
2022-07-28 17:23:50 ℹ [INFO]  Apply started.
2022-07-28 17:23:50 ℹ [INFO]  Got Backend from config: local
2022-07-28 17:23:50 ℹ [INFO]  Using dir <.devstream> to store plugins.
2022/07/28 17:24:26 StatefulSet is not ready: jenkins/dev-jenkins. 0 out of 1 expected pods are ready
2022/07/28 17:24:28 StatefulSet is not ready: jenkins/dev-jenkins. 0 out of 1 expected pods are ready
...
...
2022/07/28 17:28:39 StatefulSet is not ready: jenkins/dev-jenkins. 0 out of 1 expected pods are ready
2022/07/28 17:28:41 StatefulSet is not ready: jenkins/dev-jenkins. 0 out of 1 expected pods are ready
2022/07/28 17:28:43 StatefulSet is not ready: jenkins/dev-jenkins. 0 out of 1 expected pods are ready
2022/07/28 17:28:45 release installed successfully: dev/jenkins-4.1.13
2022-07-28 17:28:45 ℹ [INFO]  Here is how to get the password of the admin user:
2022-07-28 17:28:45 ℹ [INFO]  kubectl exec --namespace jenkins -it svc/dev-jenkins -c jenkins -- /bin/cat /run/secrets/additional/chart-admin-password && echo
2022-07-28 17:28:45 ℹ [INFO]  You are in test env. Here are the Jenkins url in host machine and Jenkins url in K8s cluster.
2022-07-28 17:28:45 ℹ [INFO]  Jenkins url in host machine: http://127.0.0.1:32000/login
2022-07-28 17:28:45 ℹ [INFO]  Jenkins url in K8s:http://172.18.0.2:32000/login
2022-07-28 17:28:45 ✔ [SUCCESS]  Tool (jenkins/default) Create done.
2022-07-28 17:28:45 ℹ [INFO]  -------------------- [  Processing progress: 2/3.  ] --------------------
2022-07-28 17:28:45 ℹ [INFO]  Processing: (jenkins-pipeline-kubernetes/default) -> Create ...
2022-07-28 17:28:45 ℹ [INFO]  credential credential-jenkins-pipeline-kubernetes-by-devstream not found, creating...
2022-07-28 17:28:46 ℹ [INFO]  job jenkins-plugin-test not found, creating...
2022-07-28 17:28:46 ✔ [SUCCESS]  Tool (jenkins-pipeline-kubernetes/default) Create done.
2022-07-28 17:28:46 ℹ [INFO]  -------------------- [  Processing progress: 3/3.  ] --------------------
2022-07-28 17:28:46 ℹ [INFO]  Processing: (jenkins-github-integ/default) -> Create ...
2022-07-28 17:28:47 ℹ [INFO]  credential credential-by-devstream-jenkins-github-integ not found, creating...
2022-07-28 17:28:48 ✔ [SUCCESS]  Tool (jenkins-github-integ/default) Create done.
2022-07-28 17:28:48 ℹ [INFO]  -------------------- [  Processing done.  ] --------------------
2022-07-28 17:28:48 ✔ [SUCCESS]  All plugins applied successfully.
2022-07-28 17:28:48 ✔ [SUCCESS]  Apply finished.

Bugs

1. About GitHub webhook creation

It seems that the GitHub webhook could not be created automatically. We need to click "save" button in the job configuration interface.

I'll try to fix it in the next pr.

2. GitHub webhook authentication warning

Git plugin 4.11.3
Lack of authentication mechanism webhook

[iyear]

Wonderful connection!