Generate explicit seed job and run it once during start-up

[jansohn]

Feature Request

I have multiple groovy job DSL scripts stored in a Git repository like this:

.
└───job-dsl
        javaapi-job-dsl.groovy
        webapps-job-dsl.groovy
        webservices-job-dsl.groovy

I would like to define only one pipeline seed job which checks out the Git repository and runs all groovy scripts it can find under the job-dsl sub-directory once during JCASC start-up phase (or when triggered manually through the UI or automatically (SCM Poll) later on).

Creating this seed job seems possible with the jobs directive but AFAICS there is no possibility to have it run once during the JCASC start-up phase, correct?

Would it make sense to add this feature or is there something fundamentally wrong in my approach?

[jetersen]

You can use

jobs:
  - script: |
    job('PROJ-unit-tests') {
        scm {
            git(gitUrl)
        }
        triggers {
            scm('*/15 * * * *')
        }
        steps {
            maven('-e clean test')
        }
    }

[jansohn]

I guess this means to run the job every 15 minutes? This would mean a lot of unnecessary runs IMHO...

Jenkins also displays a warning for this cron expression btw.

Spread load evenly by using 'H/15 * * * *' rather than '*/15 * * * *'

[AnalogJ]

we use the following queue command to trigger the DSL run after the JCasC plugin runs.

However it only seems to work 50% of the time...

- script: queue('Utility-Jobs-DSL')

After some additional research this could be related to #619

I wonder if I'm just using an older version of the JCasC plugin.

[nbutkowski]

Confirming that I also still this behavior. Running Casc plugin 1.27 and Jobs-DSL plugin 1.175 and Jenkins 2.190. For us, it consistently doesn't work. Not sure if this is something different from what @Casz fixed in #619.

Aug 20, 2019 7:43:14 PM INFO javaposse.jobdsl.plugin.JenkinsJobManagement createOrUpdateConfig
createOrUpdateConfig for seed-job
Aug 20, 2019 7:43:14 PM INFO javaposse.jobdsl.plugin.JenkinsJobManagement queueJob
Scheduling build of seed-job from seed-job
Aug 20, 2019 7:43:14 PM WARNING io.jenkins.plugins.casc.BaseConfigurator createAttribute
Can't handle class javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration#metaClass: type is abstract but not Describable.
Aug 20, 2019 7:43:14 PM WARNING io.jenkins.plugins.casc.BaseConfigurator createAttribute
Can't handle class javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration#metaClass: type is abstract but not Describable.
Aug 20, 2019 7:43:14 PM INFO jenkins.InitReactorRunner$1 onAttained
Loaded all jobs

[rmmacala]

I am also seeing this behavior. If I reapply the existing config the job will queue, but it doesn't queue on start up. This is with CasC plugin 1.31 and Job-DSL 1.76

  - script: >
      pipelineJob('unit-tests') {
        .............
     }
  - script: queue('unit-tests')   

[rmmacala]

Well I may have found a workaround that works for me. Using startup-trigger-plugin:2.9.3 and adding a trigger to my pipeline I was able to get the job to start on start up. There are still issues with trying to queue the job with - script: queue('unit-tests').

- script: >
      pipelineJob('unit-tests') {
                triggers {
                    hudsonStartupTrigger {
                      quietPeriod("60")
                      runOnChoice("ON_CONNECT")
                      label("")
                      nodeParameterName("")
                    }
                }
        .............
     }

[ChristianCiach]

@rmmacala Nice workaround, works for me! Still would be nice to see the queue declaration fixed eventually.

[ChristianCiach]

Now that jenkinsci/jenkins#4450 has been merged, does this PR help us with the wonky queue method?

[jetersen]

Sadly no as JCasC is the trigger for loading JobDSL.

I do believe we need to add something to better control when the JobDSL get's triggered to get it to trigger absolutely last.

[jetersen]

I believe #1394 will indeed fix the order of execution for Job DSL being loaded last in JCasC.

However running once is something for Job DSL to tackle as they own the configurator for JCasC. Though this can be achieved with a self removing seed job.
Feel free to create a issue over at Job DSL plugin

[jansohn]

I believe #1394 will indeed fix the order of execution for Job DSL being loaded last in JCasC.

However running once is something for Job DSL to tackle as they own the configurator for JCasC. Though this can be achieved with a self removing seed job.
Feel free to create a issue over at Job DSL plugin

@jetersen not sure I completely understand what the current status is. I just tried

  - script: >
      pipelineJob('unit-tests') {
        .............
     }
  - script: queue('unit-tests')  

which creates the job correctly but does not run it once during start-up. So basically the queue command still does not work as it seems. Do I understand correctly that this needs to be fixed in the job-dsl plugin? Maybe you can create the issue as I have no clue what needs to be fixed where...

[jetersen]

This would be an issue for the Job DSL plugin to solve.

[jansohn]

https://issues.jenkins-ci.org/browse/JENKINS-62606

[figaw]

I'm currently solving this with a seed-job that overwrites itself. Which feels like a simple solution.

You add a seed-job to your jcasc configuration which has an cron('H/2 * * * *') trigger,
and a copy of the seed-job to the repository with seed-jobs, that doesn't have the cron trigger.

You get multiple maintenance, since you have the job in two places. This is a minor issue for me since the job should literally do two things, reference a repository and trigger, and afterwards just reference the repository if I want to reload the configurations... and I'll probably never want to change this job anyways, unless I change the location of my jobdsl.

(Shameless plug) I explain in greater detail here, https://youtu.be/KB7thPsG9VA?t=1919 at 32.00.

.. But I'm looking forward to the queue thing working as well :)

[tdaniely-dn]

To workaround without maintaining script in multiple places.

jcasc:

jobs:
- providedEnv:
    RUN_AFTER_BOOT: "true"
- script: >
     {{ cat file w/e }}

seed.groovy

job("seed") {
    if (binding.hasVariable('RUN_AFTER_BOOT')) { // Only true when creating a new jenkins instance or reloading config
        triggers {
            cron("H/2 * * * *")
        }
    }
    ......
}

[nintendogameagent]

Workaround: you can trigger the seedJob by url. This call can be made by a script for example when inducing jcasc.yaml.
You can set up the build trigger and TOKEN in the seedJob configuration -> Trigger builds remotely (e.g., from scripts).
url: jenkins-url/job/seedJobs/build?token=TOKEN