-
Notifications
You must be signed in to change notification settings - Fork 0
/
Jenkins.txt
476 lines (429 loc) · 27.8 KB
/
Jenkins.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
## Apropos:
Content is versioned in git. commits, issues and pull-requests welcome!
@[https://www.github.com/earizon/DevOps]
[[{jenkins]]
#####################
# JENKINS SUMMARY/101 [[{jenkins.101,security.backup,security.secret_management,01_PM.low_code,notifications.jenkins]]
#####################
##################
# External Links #
##################
@[https://jenkins.io/doc/] @[https://jenkins.io/user-handbook.pdf]
@[https://jenkins.io/doc/book/] @[https://github.com/sahilsk/awesome-jenkins]
using/using-credentials/,
pipeline/running-pipelines,
pipeline/multibranch, Branches and Pull Requests
pipeline/docker,
pipeline/shared-libraries, Extending with Shared Libraries
pipeline/development, Pipeline Development Tools
pipeline/syntax, Pipeline Syntax
pipeline/pipeline-best-practices Pipeline Best Practices
pipeline/scaling-pipeline
blueocean/*
managing Managing Jenkins
managing/cli
managing/script-console
managing/nodes
managing/script-approval
managing/users
system-administration System Administration
## Pipeline injected ENV.VARS
- ${BASE_JENKINS_URL}/pipeline-syntax/globals#env ← FULL LIST OF ENV.VARs:
- $env.BUILD_ID :
$env.BUILD_NUMBER :
$env.BUILD_TAG : String of jenkins-${JOB_NAME}-${BUILD_NUMBER}
^^^^^^^^^^^^
Useful to subclassify resource/jar/etc output artifacts
$env.BUILD_URL : URL where results of this build can be found
(Ex.: http://buildserver/jenkins/job/MyJobName/17/)
$env.EXECUTOR_NUMBER: Unique number ID for current executor in same machine
$env.JAVA_HOME : JAVA_HOME configured for a given job
$env.JENKINS_URL :
$env.JOB_NAME : Name of the project of this build
$env.NODE_NAME : 'master', 'slave01',...
$env.WORKSPACE : absolute path for workspace
───────────────────────────────────────────────────────────────────────
# Jenkinsfile
REF: @[https://jenkins.io/doc/book/pipeline/jenkinsfile/]
@[https://jenkins.io/doc/pipeline/steps/]
## ┌ PIPELINE EXECUTION SUMMARY ──────────────────────┐
│ INPUT PROCESSING OUTPUT │
│ =========== =========== =======================│
│ Jenkinsfile → Jenkins → ✓ archived built artif.│
│ ✓ test results │
│ ✓ full console output │
│ ✓ Pipeline status │
│ (unstable,success │
│ failure,changed) │
└──────────────────────────────────────────────────┘
## ┌ DECLARATIVE SYNTAX EXAMPLE ──────┐
│ pipeline { │
│ environment { │
│ T1 = 'development' ← Env.var with global visibility
│ CC = """${sh( ← Env.var set from shell STDOUT.
│ returnStdout: true, │ trailing whitespace appended.
│ script: 'echo "clang"' │ (Use .trim() to fix)
│ )}""" │
│ AWS_SECRET_ACCESS_KEY = │
│ credentials ('aws-acc-key')← Secret protected by Jenkins [security.secret_management]
│ } │ *1
│ │
│ parameters { ← parameters allowed to be modified at runtime
│ string(name: 'Greeting', ← Referenced as ${params.Greeting}
│ defaultValue: 'Hello',│
│ description: 'Hi!') │
│ } │
│ │
│ agent any ← allocate an EXECUTOR AND WORKSPACE , it ensures
│ │ that SRC. REPO. is IMPORTED TO WORKSPACE in next stages
│ │
│ stages { ← ORDERED list of stages in Pipeline
│ │
│ stage('clone') { │
│ checkout scm ← checkout code from scm ("git clone ...") *2
│ checkout poll: false, │ scm : SPECIAL VAR. TELLING JENKINS TO
│ scm: [ │ USE SAME REPO./REVISION USED TO
│ $class: 'GitSCM', │ CHECKOUT (CLONE) JENKINSFILE
│ branches: [[name: 'dev']],
│ doGenerateSubmoduleConfigurations: false,
│ extensions: [], │
│ submoduleCfg: [], │
│ userRemoteConfigs: [ │
│ [url: 'https://github.com/user01/project01.git',
│ credentialsId: 'UserGit01']
│ ] │
│ ] │
│ } │
│ │
│ stage('Build') { ← (trans|com)pile/package/... using maven/npm/... plugins
│ │
│ environment { ← ENV.VAR with local stage visibility
│ msg1 = "Building..." │ (also visible to shell scripts)
│ EXIT = """ ← Initialized to returned status code shell exec.
│ ${sh( │ """ .... """ to embed multi-stage script.
│ returnStatus: true, │
│ script: 'exit 1' │
│ )}""" │
│ } │
│ │
│ steps { ← ORDERED list of steps in stage.
│ echo " ${msg1} :..." ← shell-like interpolation for double-coutes
│ sh 'printenv' ← msg1 and EXIT available here
│ sshagent ( │
│ crendentials: ['keys'] ←····┬─ ssh with help of agent
│ ) │ │ (ssh-agent plugin
│ { │ │ needed)
│ sh 'ssh user@remoteIP' ←····┘
│ } │
│ } │ NOTE:
│ } │ ┌─ GROOVY SYNTAX SUGAR ──────┐
│ │ │ sh([script: 'echo hello']) ← standard funct. call
│ stage('Test') { │ │ sh script: 'echo hello' ← shortcut
│ steps { ... } │ │ sh 'echo hello' ← ultra-sortcut (Single
│ } │ └────────────────────────────┘ param fun.only
│ │
│ stage('Deploy') { │
│ when { ← Conditional steps execution *1 For "complex" secrets use SNIPPET GENERATORS:
│ expression { │ ┌──────────────┬───────────────────────────────────┐
│ currentBuild.result == │SUCCESS' │ GENERATOR │ PARAMS │
│ } │ │──────────────┼───────────────────────────────────┤
│ } │ │ SSH Priv.Key │ • Key File Variable │
│ steps { sh '...' } │ │ │ • Passphrase Variable │
│ } │ │ │ • Username Variable │
│ } /* stages end */ │ │──────────────┼───────────────────────────────────┤
│ │ =============== │ Credentials │ • SSH priv/pub keys stored │
│ post { ← HANDLING ERRORS │ │ in Jenkins. │
│ always { │ =============== │──────────────┼───────────────────────────────────┤
│ junit '**/target/*.xml'│ │ (PKCS#12) │ • Keystore Variable │
│ } │ │ Certificate │ Jenkins temporary assigns it to │
│ failure { │ │ │ secure location of Cert's KeyS. │
│ mail to:team@bla.com, ← e-MAIL NOTIFICATIONS │ │ • Password Variable (Opt) │
│ subject: '...' │ [notifications.jenkins] │ │ • Alias Variable (Opt) │
│ } │ │ │ • Credentials: Cert.credentials │
│ unstable { ... } ← PIPELINE RESULT │ │ stored in Jenkins. Its value is │
│ success { ... } │ STATUS CLASSIFICATION │ │ the credential ID, which Jenkins│
│ failure { ... } │ │ │ writes out to generated snippet │
│ changed { ... } │ │──────────────┼───────────────────────────────────┤
│ } │ │ Docker Client│ • Handle Docker Host Cert.Auth. │
│ } │ │ Certificate │ │
└──────────────────────────────────┘ └──────────────┴───────────────────────────────────┘
┌ *2 GIT REPO. CLONING ("checkouts" in Jenkin) SUMMARY ────────────────┐
│ REF: @[https://jenkins.io/doc/pipeline/steps/workflow-scm-step/] │
│ checkout([ │
│ $class : 'GitSCM', │
│ poll : false, │ ┌ *1 CloneOption Class ────────────────────────────────────────────┐
│ branches : [[name: commit]], │ │· shallow (boolean) : do NOT download history (Save time/disk) │
│ extensions: [ │ │· noTags (boolean) : do NOT download tags (Save time/disk) │
│ [$class: 'RelativeTargetDirectory', relativeTargetDir: reponame],│ │ (use only what specified in refspec) │
│ [$class: 'CloneOption', reference: "/var/cache/${reponame}"] ← │· depth (int) : Set shallow clone depth (Save time/disk) │
│ ], │ │· reference(String) : local folder with existing repository │
│ submoduleCfg: [], │ │ used by Git during clone operations. │
│ userRemoteConfigs: [ │ │· timeout (int) : timeout for clone/fetch ops. │
│ [credentialsId: 'jenkins-git-credentials', url: repo_url] │ │· honorRefspec(bool): init.clone using refspec (Save time/disk)│
│ ], │ └──────────────────────────────────────────────────────────────────┘
│ doGenerateSubmoduleConfigurations: false, │
│ ]) │
└──────────────────────────────────────────────────────────────────────┘
# ┌─ MULTIAGENT PIPELINES COMMON BUILD ──┐ Useful for multi─target builds/tests/...
│ pipeline { │ reusing common builds in different agents.
│ agent none │
│ stages { │
│ stage('clone') { ... } │
│ stage('Build') { │
│ agent any │
│ steps { │
│ ... │
│ stash ([ │
│ includes: '**/target/*.jar',│
│ name: 'app' ]) ←··┐
│ } │ ·
│ } │ ·
│ stage('Linux') { │ · copy named─stash FROM JENKINS MASTER TO CURRENT WORKSP.
│ agent { label 'linux' } │ · NOTE: stash = something put away for future use
│ steps { │ · (In practice: Named cache of generated artifacts
│ unstash 'app' ←··• during same pipeline for reuse in further steps)
│ sh '...' │ · Removed at pipeline termination.
│ } │ ·
│ post { ... } │ ·
│ } │ ·
│ stage('Test on Windows') { │ ·
│ agent { label 'windows' } │ ·
│ steps { │ ·
│ unstash 'app' ←··┘
│ bat '...' │
│ } │
│ post { ... } │
│ } │
│ } │
│ } │
└──────────────────────────────────────┘
# ┌─ PARALLEL EXECUTION ───────────┐
│ stage('Test') { │
│ parallel ←·• Execute linux&windows nodes
│ linux: { │ · in parallel.
│ node('linux') { ←─┤
│ try { │ ·
│ unstash 'app' │ ·
│ sh 'make check' │ ·
│ } │ ·
│ finally { │ ·
│ junit '**/target/*.xml'│ ·
│ } │ ·
│ } │ ·
│ }, │ ·
│ windows: { │ ·
│ node('windows') { ... } ←·┘
│ } │
│ } │
└────────────────────────────────┘
[[jenkins.101}]]
# Jenkins Pipeline Builder [[{01_PM.low_code,QA.UX]]
@[https://jenkinspipelinegenerator.octopus.com/#/]
https://www.linuxtoday.com/developer/jenkins-pipeline-builder-now-available/
pipeline builder scans your public GitHub repository and attempts
to build a custom Jenkins Pipeline that you can then copy and paste
into a Jenkins Pipeline project.
Support for maven, php, node,python, .Net Core, ..
[[}]]
────────────────────────────────────────────────────────────────────────────────
# Admin 101: EXPORT/IMPORT jobs [[{security.backup]]
REF: @[https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI]
└ * ALT 1: Using jenkins-cli.jar
******************************
· PRE-REQUISITES)
· jenkins-cli.jar version must match Server version
· jnlp ports need to be open
· Script:
JENKINS_CLI="java -jar ${JENKINS_HOME}/war/WEB-INF/jenkins-cli.jar -s ${SERVER_URL}"
${JENKINS_CLI} get-job job01 > job01.xml ← job01.xml can be "gitted",...
${JENKINS_CLI} create-job job01 < job01.xml
WARN: There are issues with bare naked ampersands in the XML such as when you have & in Groovy code.
(REF: @[https://stackoverflow.com/questions/8424228/export-import-jobs-in-jenkins])
└ · ALT 2: USING CURL
*******************
SERVER_URL = "http://..." ← Without Authentication
SERVER_URL = "http://${USER}:${API_TOKEN}@..." ← With Authentication
$ curl -s ${SERVER_URL}/job/ JOBNAME/config.xml > job01.xml ← EXPORT
$ curl -X POST ${SERVER_URL}/createItem?name=JOBNAME' \ ← IMPORT
--header "Content-Type: application/xml" -d job01.xml
└ ALT 3: Filesystem (backup)
**************************
$ tar cjf _var_lib_jenkins_jobs.tar.bz2 /var/lib/jenkins/jobs
[[security.backup}]]
# Admin 101: Dockerized Jenkins [[{01_PM.low_code]]
$ docker run --rm -u root -p 8080:8080 \
-v jenkins-data:/var/jenkins_home \ ← if 'jenkins-data' Docker volumen
\ doesn't exists it will be created
\
-v /var/run/docker.sock:/var/run/docker.sock \ ← Jenkins need control of Docker to
\ launch new Docker instances during
\ the build process
-v "$HOME":/home \
--name jenkins01 \ ← Allows to "enter" docker with:
jenkinsci/blueocean $ docker exec -it jenkins01 bash
[[}]]
# End-to-End Multibranch Pipeline [[{jenkins.101,01_PM.TODO]]
@[https://jenkins.io/doc/tutorials/build-a-multibranch-pipeline-project/]
PREREQUISITES: Git installed, Docker installed.
┌──────────────┬────────────┬────────────┐
│ INPUT → JENKINS → OUTPUT │
│ ARTIFACTS → → ARTIFACTS │
├──────────────┼────────────┼────────────┤
│ Node.js │ build→test │ development│
│ React app │ │ production │
│ npm │ │ │
└──────────────┴────────────┴────────────┘
STEP 1) Setup local git repository
- clone:
$ git clone https://github.com/?????/building-a-multibranch-pipeline-project
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Forked from
https://github.com/jenkins-docs/building...
- Creak dev/pro branches:
$ git branch development
$ git branch production
STEP 2) Add 'Jenkinsfile' stub (agent, stages sections) to repo
(initially in master branch)
STEP 3) Create new Pipeline in Jenkins Blue Ocean Interaface
browse to "http://localhost:8080/"
→ click "Create a new Pipeline"
→ Choose "Git" in "In Where do you store your code?"
→ Repository URL: "/home/.../building-a-multibranch-pipeline-project"
→ Save
Blue Ocean will detect the presence of the "Jenkinsfile" stub
in each branch and will run each Pipeline against its respective branch,
STEP 4) Start adding functionality to the Jenkinsfile pipeline
(commit to git once edited)
pipeline {
environment {
docker_caching = 'HOME/.m2:/root/.m2' ← Cache to speed-up builds
docker_ports = '-p 3000:3000 -p 5000:5000' ← Cache to speed-up builds
}
agent {
docker {
image 'node:6-alpine' ← Good Enough to build simple
Node.js+React apps
args '' ← dev/pro port where the app will
} listen for requests. Used during
} functional testing
environment {
CI = 'true'
}
stages {
stage('Build') {
steps {
sh 'npm install' ← 1st real build command
}
}
stage('Test') {
steps {
sh './jenkins/scripts/test.sh'
}
}
}
}
STEP 5) Click "run" icon of the master branch of your Pipeline project,
and check the result.
STEP 6) Add "deliver" and "deploy" stages to the Jenkinsfile Pipeline
(and commit changes)
Jenkins will selectively execute based on the branch that Jenkins is building from
+ stage('Deliver for development') {
+ when {
+ branch 'development'
+ }
+ steps {
+ sh './jenkins/scripts/deliver-for-development.sh'
+ input message: 'Finished using the web site? (Click "Proceed" to continue)'
+ sh './jenkins/scripts/kill.sh'
+ }
+ }
+ stage('Deploy for production') {
+ when {
+ branch 'production'
+ }
+ steps {
+ sh './jenkins/scripts/deploy-for-production.sh'
+ input message: 'Finished using the web site? (Click "Proceed" to continue)'
+ sh './jenkins/scripts/kill.sh'
+ }
+ }
[[}]]
# Jenkins Unordered: [[{01_PM.TODO]]
## Example Pipeline script:[[{jenkins.101,01_PM.TODO]]
@[https://jenkins.io/doc/pipeline/steps/pipeline-build-step/]
build job: 'Pipeline01FromJenkinsfileAtGit', propagate: true, wait: false
build job: 'Pipeline02FromJenkinsfileAtGit', propagate: true, wait: false
build job: 'Pipeline03FromJenkinsfileAtGit', propagate: true, wait: false
^^^^
result of step is that of downstream build
(success, unstable, failure, not built, or aborted).
false → step succeeds even if the downstream build failed
use result property of the return value as needed.
[[}]]
## Serverless Pipeline [[{jenkins.101,jenkins.troubleshooting,qa.testing,01_PM.TODO]]
Jenkinsfile-runner: Exec. Jenkinsfile pipeline without running Jenkin server.
- https://jenkins.io/blog/2019/02/28/serverless-jenkins/
- https://medium.com/@jdrawlings/serverless-jenkins-with-jenkins-x-9134cbfe6870
@[https://github.com/jenkinsci/jenkinsfile-runner]
[[}]]
## AWS EC2 plugin: launch Amazon EC2 Spot Instances as worker nodes [[cloud.aws]]
automatically scaling capacity with load demand.
@[https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Fleet+Plugin]
## "Organization Folders" allows Jenkins to monitor entire [[git.github,git.bitbucket,git.gerrit]]
GitHub Organization or BitbucketTeam/Project and automatically create new
Multibranch Pipelines for multi-branch repos by pulling Jenkinsfile's.
## Jenkins + Zuul integration: @[https://zuul-ci.org/]
• Zuul is a python daemon listening for Gerrit stream-events feed and
trigger jobs function registered by Jenkins using the Jenkins Gearman
plugin. triggers are set with YAML and hosted in the git repo.
integration/config.git as /zuul/layout.yaml.
@[https://www.mediawiki.org/wiki/Continuous_integration/Zuul]
· IBM OpenStack Engineer Urges Augmenting Jenkins with Zuul for Hyperscale Projects
@[https://thenewstack.io/ibm-openstack-engineer-urges-cncf-consider-augmenting-jenkins-zuul/]
## Jenkins X: https://jenkins-x.io/es/
Accelerate Continuous Delivery on Kubernetes [[k8s]]
· https://github.com/kurron/jx3-k3s-vault
Jenkins X 3.x GitOps repository using k3s to create a kubernetes
cluster, github for the git and container registry and external vault
· Rather than having to have deep knowledge of the internals of
Jenkins X Pipeline, Jenkins X will default awesome pipelines for your
projects that implements fully CI and CD."
· Environment Promotion via GitOps.
· Preview Environments.
· Feedback on Issues and Pull Requests.
## Customize History Saving Policy:[[{jenkins.101,jenkins.build_history]]
@[https://stackoverflow.com/questions/60391327/is-it-possible-in-jenkins-to-keep-just-first-and-last-failures-in-a-row-of-con]
Use Case: We are just interesing in keeping "build" changes when the execution
changes from "success execution" to "failure". That's is, if w have a history like:
t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15
-----------------------------------------------------------
OK, OK, OK, OK, KO, KO, KO, KO, OK, OK, OK, OK, KO, KO, OK
^ ^ ^ ^ ^
status status status status status
change change change change change
We want to keep history just for:
t1 t5 t9 t13 t15
-----------------------------------------------------------
OK, KO, OK, KO, OK
To allow this history saving policy a groovy job-post-build step is needed:
Ex: discard all successful builds of a job except for the last 3 ones
(since typically, you're more interested in the failed runs)
def allSuccessfulBuilds = manager.build.project.getBuilds().findAll {
it.result?.isBetterOrEqualTo( hudson.model.Result.SUCCESS )
}
allSuccessfulBuilds.drop(3).each {
it.delete()
}
[[}]]
## build-status conditional Post-build actions: [[{jenkins.101,git,performance,01_PM.TODO}]]
@[https://stackoverflow.com/questions/45456564/jenkins-declarative-pipeline-conditional-post-action]
## Clone directovy (vs full repo) [[performance]]
@[https://softwaretestingboard.com/q2a/1791/how-clone-checkout-specific-directory-command-line-jenkins]
Ex:
$ git checkout branch_or_version -- path/file
$ git checkout HEAD -- main.c ← checkout main.c from HEAD
$ git checkout e5224c883a...c9 /path/to/directory ← Checkout folder from commit
## Jenkins: managing large git repos [[performance]]
@[https://jenkins.io/files/2016/jenkins-world/large-git-repos.pdf]
[[01_PM.TODO}]]
[[jenkins}]]