Skip to content

Latest commit

 

History

History
159 lines (130 loc) · 8.6 KB

README.md

File metadata and controls

159 lines (130 loc) · 8.6 KB

Eclipse OMR Jenkins Builds

This folder contains Jenkins pipeline scripts that are used in the OMR Jenkins builds.

Setting up a Jenkins job

Merge Builds

From the Builds view, on the left menu select New Item. Name the job based on the following convention Build-<SPEC> (eg. Build-linux_x86-64. See omrbuild.groovy for full SPEC list). Select Pipeline as the job type and then click OK. Setup the following in the job config.

  1. General
    1. Discard old builds -> Max # of builds to keep: 25
    2. GitHub project -> Project url: https://github.com/eclipse/omr/
    3. This project is parameterized -> Choice Parameter -> Name: BUILDSPEC -> Choices: <SPEC> (The matching SPEC name)
  2. Build Triggers
    1. Generic Webhook Trigger (defaults should be fine)
  3. Pipeline
    1. Definition: Pipeline script from SCM
    2. SCM: Git
      1. Repositories -> Repository URL: https://github.com/eclipse/omr.git
      2. Branches to build: /refs/heads/master
    3. Script Path: buildenv/jenkins/omrbuild.groovy
    4. Lightweight checkout: true

Pull Request Builds

From the Pull Requests view, on the left menu select New Item. Name the job based on the following convention PullRequest-<SPEC> (eg. PullRequest-linux_x86-64. See omrbuild.groovy for full SPEC list). Select Pipeline as the job type and then click OK. Setup the following in the job config.

  1. General
    1. GitHub project -> Project url: https://github.com/eclipse/omr/
    2. This project is parameterized -> Choice Parameter -> Name: BUILDSPEC -> Choices: <SPEC> (The matching SPEC name)
  2. Build Triggers
    1. GitHub Pull Request Builder
      1. Admin list: <Committers' Github IDs> (Should be auto filled from Global Config)
      2. Use github hooks for build triggering: true
      3. Trigger phrase: .*Jenkins build.*(aix|all).* (Replace aix with spec shorthand. Where aix is a shorthand for aix_ppc-64)
      4. White list: <List of Github IDs allowed to launch PR builds>
      5. Trigger Setup
        1. Add -> Update commit status during build -> Commit Status Context: continuous-integration/eclipse-omr/pr/<SPEC>
        2. Add -> Cancel build on update
  3. Pipeline
    1. Definition: Pipeline script from SCM
    2. SCM: Git
      1. Repositories
        1. Repository URL: https://github.com/eclipse/omr.git
        2. Advanced -> Refspec: +refs/pull/${ghprbPullId}/merge:refs/remotes/origin/pr/${ghprbPullId}/merge
      2. Branches to build: ${sha1}
      3. Additional Behaviours
        1. Clean before checkout
        2. Advanced clone behaviours
          1. Fetch tags: false
          2. Honor refspec on initial clone: true
    3. Script Path: buildenv/jenkins/omrbuild.groovy
    4. Lightweight checkout: false

Configuring the Github repo hooks

Eclipse admins must do this via a Bugzilla request. Product: Community, Component: GitHub

  1. Generic Webhook for Merge builds
    Payload URL: https://ci.eclipse.org/omr/generic-webhook-trigger/invoke
    Content type: application/json
    Enable SSL verification: true
    Which events would you like to trigger this webhook?: Just the push event
    Active: true

  2. Github PullRequest Trigger Webhook
    Payload URL: https://ci.eclipse.org/omr/ghprbhook/
    Content type: application/x-www-form-urlencoded
    Enable SSL verification: true
    Which events would you like to trigger this webhook?: Let me select individual events -> Issue comments, Pull requests, Pushes
    Active: true

Pipeline script from SCM subtleties with Rocket Git for z/OS

Configuring Jenkins to pull the pipeline scripts from Git on z/OS is a non-trivial effort. The Rocket Git for z/OS [1] port does not support https transport protocol, thus specifying the repository in the Repository URL with an https:// prefix will not work as Jenkins will encounter errors when trying to execute commands on the slave:

Fetching changes from the remote Git repository
Cleaning workspace
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/eclipse/omr.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/eclipse/omr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: fatal: Unable to find remote helper for 'https'

Thus we must use the git:// prefix to specify the Repository URL in the Jenkins Git Plugin. This is a supported configuration [2]. However one may run into issues when using the git:// transport protocol when attempting to launch a Jenkins build on z/OS:

hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/master:refs/remotes/origin/master --prune" returned status code 128:
stdout:
stderr: fatal: unable to connect to github.com:
github.com[0: 192.30.253.113]: errno=Connection timed out


	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2042)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1761)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:72)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:442)
	at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:351)
	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:198)
	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:174)
	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:108)
	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

The reason this happens is because the native git:// transport uses TCP port 9418 which may not be open on the Jenkins master, hence why we end up timing out. To fix this there are two options:

  1. Open port 9418 on the Jenkins master

  2. Alias git:// transport protocol to https:// [3]

    This can be achieved by navigating to the master node Script Console in Jenkins and executing the following Groovy script:

    println "git config --global url.https://.insteadOf git://".execute().text

[1] https://www.rocketsoftware.com/product-categories/mainframe/git-for-zos [2] https://wiki.jenkins.io/display/JENKINS/Git+Plugin [3] angular/angular-phonecat#141 (comment)