Declarative Pipeline support #47

MichalPloski · 2018-03-16T13:56:54Z

It would be nice to have a chance to use this plugin with declarative pipeline under trigger sections.
In my case we have all pipelines written in declarative way except one using generic trigger.

berkesokhan · 2018-05-01T20:24:01Z

At least can we use any workaround to incorporate this plugin to Declarative Pipelines? (e.g. script block?)

tomasbjerre · 2018-05-02T15:56:19Z

The problem here is documentation. And the fact that I cannot find any documentation at all on how to do this.

If anyone can point me to a plugin that supports it, then I can probably figure out how it should be done!

I use the plugin in 2 ways.

Pipeline jobs. Where I create the job from Job DSL and that is where I configure the plugin.
Multibranch pipeline. Where I configure the plugin with pipeline, that is in the readme.

tomasbjerre · 2018-05-05T19:16:48Z

Should be fixed now in 1.33. Open issue again if not working.

berkesokhan · 2018-05-05T19:44:40Z

Would you mind sharing a usage snippet under a declarative pipeline's triggers section so we may try?

tomasbjerre · 2018-05-05T20:53:08Z

I added an example now.

berkesokhan · 2018-05-05T21:12:52Z

Thanks! Saw the new README example, I will report if we found any errors.

ntindall · 2018-06-05T00:11:17Z

I'm not able to get the README example working @tomasbjerre -- I copied it verbatim. @berkesokhan, were you able to get this working?

The webhook successfully triggers the job... but I can't get any of the JSON parsing / extraction logic to work... any ideas? I pasted the log output, but its not logging the curl request body, which is making me think it may not be processing it for some reason?

curl -vs -X POST https://$JENKINS_URL/generic-webhook-trigger/invoke -d '{
  "before": "1848f1236ae15769e6b31e9c4477d8150b018453",
  "after": "5cab18338eaa83240ab86c7b775a9b27b51ef11d",
  "ref": "refs/heads/develop"
}'

pipeline {
  // Restrict the jenkins machine that this pipeline is allowed to run on.
  agent {
    label 'Jenkins-Docker-Slave'
  }

  triggers {
    GenericTrigger(
      genericVariables: [
        [key: 'ref', value: '$.ref']
      ],
      causeString: 'Triggered on $ref',
      regexpFilterExpression: '',
      regexpFilterText: '',
      printContributedVariables: true,
      printPostContent: true
    )
  }

  environment {
    REPO = 'pdaas'
  }

  stages {
    stage('build_image') {
      steps {
        sh '''
        echo Variables from shell:
        echo ref $ref
        '''
      }
    }
  }
}

here is the output:

Triggered on $ref
 > git rev-parse --is-inside-work-tree # timeout=10
Setting origin to git@github.plaid.com:plaid/leeroy.git
 > git config remote.origin.url git@github.plaid.com:plaid/leeroy.git # timeout=10
Fetching origin...
Fetching upstream changes from origin
 > git --version # timeout=10
using GIT_SSH to set credentials 
 > git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/*
Seen branch in repository origin/dam-adding-support-for-nvmrc
Seen branch in repository origin/dc-dromedary
Seen branch in repository origin/jkim-2017.09.19-add-monitoring
Seen branch in repository origin/jkim-2017.12.18-moar-configurability
Seen branch in repository origin/jkim-2018.03.11-add-BUILD_COMMIT
Seen branch in repository origin/lr-cancel-deploy
Seen branch in repository origin/lr-sha-issue
Seen branch in repository origin/master
Seen branch in repository origin/mk-intuit
Seen branch in repository origin/mk-intuit-api-sandbox-tests
Seen branch in repository origin/mk-longtail
Seen branch in repository origin/nk-add-escalate
Seen branch in repository origin/nk-dromedary-always-production
Seen branch in repository origin/nt-ISO
Seen branch in repository origin/nt-Jenkinsfile-gd2dEq
Seen branch in repository origin/nt-merge_branches
Seen branch in repository origin/nt-monodeploy-RS9GyJ
Seen branch in repository origin/nt-poll-more-intelligently-WJfdo8
Seen branch in repository origin/pc-opsworks-rollback
Seen branch in repository origin/testing-123
Seen 20 remote branches
Obtained pipelines/deployments/pdaas.Jenkinsfile from adc37bbb3a5e1a9c1957c46d65f76eac4c616c01
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
GenericWebhookEnvironmentContributor
 Received:




Contributing variables:



Running on jenkins-deploy-slave-ue1c-3 in /data/jenkins/workspace/workspace/pdaas_nt-Jenkinsfile-gd2dEq-WFRXNAPMJDCMZBAOYAAXY3PQXUGVVVNPIJEGV6NMMSOMWJYYS7CQ
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git@github.plaid.com:plaid/leeroy.git # timeout=10
Fetching without tags
Fetching upstream changes from git@github.plaid.com:plaid/leeroy.git
 > git --version # timeout=10
using GIT_SSH to set credentials 
 > git fetch --no-tags --progress git@github.plaid.com:plaid/leeroy.git +refs/heads/*:refs/remotes/origin/*
Checking out Revision adc37bbb3a5e1a9c1957c46d65f76eac4c616c01 (nt-Jenkinsfile-gd2dEq)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f adc37bbb3a5e1a9c1957c46d65f76eac4c616c01
Commit message: "no build"
 > git rev-list --no-walk cf804c5ff620276176a2b74ad8c9295b9aea4549 # timeout=10
First time build. Skipping changelog.
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build_image)
[Pipeline] sh
[pdaas_nt-Jenkinsfile-gd2dEq-WFRXNAPMJDCMZBAOYAAXY3PQXUGVVVNPIJEGV6NMMSOMWJYYS7CQ] Running shell script
+ echo Variables from shell:
Variables from shell:
+ echo ref
ref
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

ntindall · 2018-06-05T01:23:29Z

I figured it out -- the webhook expects for the Content-Type header to be set to application/json 🤦 (I was omtting it....)

It would be good to add some validation that errors more loudly when this happens

albertinisg · 2018-11-07T11:35:19Z

I'm following the documentation and I'm facing a weird issue with the declarative pipeline. If I configure the plugin using the interface, everything works and I get the right output:

{
    "status": "ok",
    "data": {
        "triggerResults": {
            "ci-playground": {
                "id": 38751,
                "regexpFilterExpression": "",
                "regexpFilterText": "",
                "resolvedVariables": {},
                "triggered": true,
                "url": "queue/item/38751/"
            }
        }
    }
}

But if I follow the documentation example for the declarative pipeline, the job is never triggered. I should be missing something simple, but can't find out what...

Plugin version: 1.46

My pipeline is declared as:

pipeline {
  agent any
  triggers {
    GenericTrigger(
     genericVariables: [
      [key: 'ref', value: '$.ref']
     ],
     
     printContributedVariables: true,
     printPostContent: true
    )
  }
  stages {
    stage('Some step') {
      steps {
        sh "echo $ref"
      }
    }
  }
}

So then I call it like:

$ curl -X POST -H "Content-Type: application/json" -d '{ "before": "1848f12", "after": "5cab1", "ref": "refs/heads/develop" }' --insecure -v "https://<user>:<pass>@<myjenkinsinstance>/generic-webhook-trigger/invoke"

And I get a 200 OK, but the job doesn't start:

> POST /generic-webhook-trigger/invoke HTTP/1.1
> Host: XXXXXXXXXXXXXXXX
> Authorization: Basic XXXXXXXXXXXXXXX
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 70
> 
* upload completely sent off: 70 out of 70 bytes
< HTTP/1.1 200 OK
< Server: nginx/1.13.10
< Date: Wed, 07 Nov 2018 11:30:17 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Content-Type-Options: nosniff

What am I missing?

tomasbjerre · 2018-11-07T11:42:26Z

Might be same as #84 and/or #81 but they were never really resolved.

You could try adding a token and supply that in your request. But even if that works it is still strange because the plugin should have responded with a clear message telling you to do that: https://github.com/jenkinsci/generic-webhook-trigger-plugin/blob/master/src/main/java/org/jenkinsci/plugins/gwt/GenericWebHookRequestReceiver.java#L33

You comment contains < Server: nginx/1.13.10 which is interesting. Are you sure Jenkins is being invoked? Perhaps it is a proxy responding with HTTP 200.

albertinisg · 2018-11-07T11:59:36Z

Thanks for the superquick response!

Modified the pipeline to add the token:

pipeline {
  agent any
  triggers {
    GenericTrigger(
     genericVariables: [
      [key: 'ref', value: '$.ref']
     ],
     
     token: 'abc123',
     
     printContributedVariables: true,
     printPostContent: true
    )
  }
  stages {
    stage('Some step') {
      steps {
        sh "echo $ref"
      }
    }
  }
}

Then triggered providing the token with the 3 different ways possible: as a parameter, in the header and bearer token; no success.

About the nginx, if I configure the plugin with the interface I get it to work even with the nginx in between:

> POST /generic-webhook-trigger/invoke HTTP/1.1
> Host: XXXXXXXXXXXXXXXXXX
> Authorization: Basic XXXXXXXXXXXXXXXXXXXX
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 70
> 
* upload completely sent off: 70 out of 70 bytes
< HTTP/1.1 200 OK
< Server: nginx/1.13.10
< Date: Wed, 07 Nov 2018 11:48:54 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 231
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< 
{"status":"ok","data":{"triggerResults":{"ci-playground":{"id":38758,"regexpFilterExpression":"","regexpFilterText":"","resolvedVariables":{"ref":"refs/heads/develop

Is there any scenario where the plugin returns just a 200 OK? Otherwise there might be something with the nginx yes... But still weird that it works with the interface and not the pipeline itself

tomasbjerre · 2018-11-07T12:11:16Z

What do you mean with "if I configure the plugin with the interface"? Is that by using the GUI?

And when using the pipeline, do you use multibranch pipeline plugin? Did you make sure the multibranch job has been launched to synk with the Git repositories?

If not using multibranch pipeline, I'm not really sure how that works. I guess the properties are applied to a job when the pipeline is run.

Anyway, when using properties from a pipeline, you may browse to the jobs and check what configuration has been applied to them. By simply viewing the jobs configuration in the GUI.

I usually script all jobs with Job DSL plugin and let Job DSL configure this plugin. The jobs created by Job DSL are pipelines and I have no configuration of this plugin in my pipelines.

tomasbjerre · 2018-11-07T13:59:21Z

Releasing 1.47 now with a fix. So that the plugin will print information when no jobs were found and not be silent. Still that does not solve the actual issue for you.

albertinisg · 2018-11-08T11:28:36Z

What do you mean with "if I configure the plugin with the interface"? Is that by using the GUI?

Yes, I meant the GUI.

And when using the pipeline, do you use multibranch pipeline plugin? Did you make sure the multibranch job has been launched to synk with the Git repositories?

I was not using the multibranch pipeline plugin and I use it as a freestyle project, as I just want to integrate it with an external tool to trigger the pipeline.

If not using multibranch pipeline, I'm not really sure how that works. I guess the properties are applied to a job when the pipeline is run.

I was just expecting it to behave as any API: when a POST is received, it checks the variable for the trigger and if it matches, it triggers the pipeline.

I didn't know that this was ment to be used with the Multibranch pipeline. I might try with the Job DSL plugin to see how it works! :)

Thanks again for your support

albertinisg · 2018-11-08T11:30:28Z

Releasing 1.47 now with a fix. So that the plugin will print information when no jobs were found and not be silent. Still that does not solve the actual issue for you.

Just seen that :D thanks a lot!!

albertinisg · 2018-11-13T10:05:44Z

I did test it with the Job DSL with no success either :(

Also, now with the same pipeline I had before, I can see more information in the response, which is great!

{
    "status": "ok",
    "data": {
        "triggerResults": {
            "ANY": "Did not find any jobs with GenericTrigger configured! If you are using a token, you need to pass it like ...trigger/invoke?token=TOKENHERE. If you are not using a token, you need to authenticate like http://user:passsword@jenkins/generic-webhook... "
        }
    }
}

Anyway, I was just trying to set it up for a simple example and just filter as explained in the doc, but couldn't make it work through the GUI either. With the following configuration, it triggers the build no matter the body of the POST. The plugin is configured as follows and I set the regexp as in the Gherkin test in https://github.com/jenkinsci/generic-webhook-trigger-plugin/blob/master/src/test/resources/org/jenkinsci/plugins/gwt/bdd/github/github-push-tag.feature:

After that, I make the same request with different body:

{ "before": "1848f12", "after": "5cab1", "ref": "refs/tags/develop" }
{ "before": "1848f12", "after": "5cab1", "ref": "refs/heads/develop" }

And both requests starts the build:

{
    "status": "ok",
    "data": {
        "triggerResults": {
            "ci-playground": {
                "id": 39047,
                "regexpFilterExpression": "",
                "regexpFilterText": "",
                "resolvedVariables": {},
                "triggered": true,
                "url": "queue/item/39047/"
            }
        }
    }
}

To me it looks interesting that the fields regexpFilterExpression, regexpFilterText and resolvedVariables are empty. They shouldn't be, right?

tomasbjerre · 2018-11-13T10:34:43Z

Perhaps you did not mean to have that advanced regexp in the value filter of the ref parameter? The text just below the field explains what it does.

And you have other field below, not shown inte your screenshot, for regexpFilterExpression and regexpFilterText .

albertinisg · 2018-11-13T12:39:26Z

Damn... you are 100% right... I added the regular expression in regexpFilterExpression and the variable in regexpFilterText and I can see it now in the response and it blocks the triggering:

{
    "status": "ok",
    "data": {
        "triggerResults": {
            "ci-playground": {
                "id": 0,
                "regexpFilterExpression": "^(refs/tags/.+)$",
                "regexpFilterText": "$ref",
                "resolvedVariables": {},
                "triggered": false,
                "url": ""
            }
        }
    }
}

Thanks a lot for all your patience and support!! :)

tomasbjerre added the enhancement label Mar 18, 2018

tomasbjerre added a commit that referenced this issue May 5, 2018

Declarative Pipeline support #47

9d29aa9

tomasbjerre closed this as completed May 5, 2018

tomasbjerre added a commit that referenced this issue Nov 7, 2018

Printing information when no jobs found #47 #84 #81

800def9

tomasbjerre mentioned this issue Mar 27, 2019

How to use withCredentials in declarative Jenkinsfile? #107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Declarative Pipeline support #47

Declarative Pipeline support #47

MichalPloski commented Mar 16, 2018

berkesokhan commented May 1, 2018

tomasbjerre commented May 2, 2018

tomasbjerre commented May 5, 2018

berkesokhan commented May 5, 2018

tomasbjerre commented May 5, 2018

berkesokhan commented May 5, 2018 •

edited

Loading

ntindall commented Jun 5, 2018 •

edited

Loading

ntindall commented Jun 5, 2018

albertinisg commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

albertinisg commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

albertinisg commented Nov 8, 2018

albertinisg commented Nov 8, 2018

albertinisg commented Nov 13, 2018

tomasbjerre commented Nov 13, 2018

albertinisg commented Nov 13, 2018

Declarative Pipeline support #47

Declarative Pipeline support #47

Comments

MichalPloski commented Mar 16, 2018

berkesokhan commented May 1, 2018

tomasbjerre commented May 2, 2018

tomasbjerre commented May 5, 2018

berkesokhan commented May 5, 2018

tomasbjerre commented May 5, 2018

berkesokhan commented May 5, 2018 • edited Loading

ntindall commented Jun 5, 2018 • edited Loading

ntindall commented Jun 5, 2018

albertinisg commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

albertinisg commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

tomasbjerre commented Nov 7, 2018

albertinisg commented Nov 8, 2018

albertinisg commented Nov 8, 2018

albertinisg commented Nov 13, 2018

tomasbjerre commented Nov 13, 2018

albertinisg commented Nov 13, 2018

berkesokhan commented May 5, 2018 •

edited

Loading

ntindall commented Jun 5, 2018 •

edited

Loading