Skip to content

Jenkins

Jump to bottom
ILLYAKO edited this page Dec 14, 2022 · 22 revisions

Jenkins

https://www.youtube.com/watch?v=pMO26j2OUME&list=PLy7NrYWoggjw_LIiDK1LXdNN82uYuuuiC
jenkins-docker-app

  1. Jenkins Container Setup
  2. Create Multibranch Pipeline with Git Repo
  3. Types of Credentials
  4. Jenkinsfile
    Code -> Commit-> Git Repo-> Trigger build -> Jenkinsfile:test, build, deploy

1. Jenkins Container Setup

Run Jenkins in Docker Container

1.1. Find a Jenkins image in dockerhub

The image 'jenkins/jenkins' should be used.

1.2. Docker run option

  • Expose port 8080 -- By default runs on that port
  • Expose port 50000 -- Master/Slaver Communication
  • Run in detached mode -- run container in the background
  • Bind named volume -- persist data of Jenkins, bind host directory and container directory
  • image version -- latest (jenkins/jenkins:lts)
    .

1.3. Run docker container in CLI

docker run -p 8080:8080 -p 50000:50000 -d -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts

1.3.1. Docker Container list

docker ps

1.3.2. Docker logs

docker logs your_ContainerID

1.3.2.1 An admin user has been created and a password generated

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

3adc1a26a8fd4128a816ca8ba01da498

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

1.3.3. Open browser localhost:8080 and insert the admin password from the log

1.3.4. Install suggested plugins

1.3.5. Create the First Admin User

IlLya 1234 Illya K my@email.com Jenkins URL http://localhost:8080/ Save and Finish

1.3.6. + New Item

  • Freestyle: simple, single tasks e.g. run tests
  • Pipeline: whole delivery cycle e.g. test|build|... for a single branch
  • Multibranch Pipeline : like pipeline for multiple branches

1.3.7. Go to Dashboard -> Manage Jenkins -> Manage Plugins

Updates Available plugins Installed plugins Advanced settings Download progress

2. Create Multibrach Pipeline with Git

2.1 + New Item

Name: "my-pipeline"

2.2. Multibranch Pipeline

Branch Source = Project you want to build
Add Source -> Git
Project Repository - Specify the URL of this remote repository. This uses the same syntax as your git clone command HTTPS.
https://github.com/ILLYAKO/jenkins-docker-app.git
Credentials - > choose created credentials for git repo.
Behaviours -> Add -> Behaviors: Discover branches
Filter by name (with regular expression) -> .* (default means all branches)

Build Configuration Mode -> by Jenkins(default)
Script Path -> Jenkinsfile(default)
Save

2.2.1 Jenkins Credentials

"Credentials" Plugin to store and manage them centrally
Dashboards -> Manage Jenkins -> Manage Credentials -> Credentials -> Jenkins System(Store) -> global(Domains) -> Add Credentials

Kinds:
-Username with password
-GitHub App
-SSH Username with private key
-Secret file
-Secret text
-Certificate

Credential Scopes:
-System: Only available on Jenkins server NOT for Jenkins jobs
-Global: Everywhere accessible
-Project: Limited to project, ONLY with multibranch pipeline (on project page)
ID = Reference for your credentials

2.2.1.1 Create Jenkins Global Credentials

Kind: User with password
Scope: Global (Jenkins, nodes, items, all child items, etc)
Username: global
Password: 1234
ID: global

2.2.1.1 Create Jenkins System Credentials

Kind: User with password
Scope: System (Jenkins and nodes only)
Username: system
Password: 1234
ID: system

2.2.2. Go to the pipeline "my-pipeline" git credentials

kind: username with password
Username: my@email.com
password: *******
ID: demo-app-git-credentials

2.2.3. Discover branches by name

Go Dashboard -> my-pipeline -> Configuration -> Branch Sources -> Behaviours -> Add
Behaviors: Discover branches
Filter by name(with regular expression):^dev|master|feature.*$

3. Basic Jenkinsfile

3.1. Pipeline Syntax

  • Scripted -first syntax
    -Groovy engine
    -Advanced scripting capabilities, high flexibility
    -difficult to start
  • Declarative -recent addition -easier to get started, but not that powerful -pre-defined structure

3.2. Required Fields of Jenkinsfile

-'pipeline' must be top-level -'agent' -where to execute -'stages' -where the 'work' happens -'stage' and 'steps' -'step' -where execute a command in the Jenkins server

pipeline {
    agent any
    stages {
        stage("build") {
            steps {
                echo 'building the application...'
            }
        }
        stage("test") {
            steps {
                echo 'testing the application...'
            }
        }
        stage("deploy") {
            steps {
                echo 'deploying the application...'
            }
        }
    }
}

3.2.1. Scan Multibranch Pipeline Now

Dashboard -> my-pipeline -> Scan Multibranch Pipeline Now

3.2.2. Scan Multibranch Pipeline Log

Dashboard -> my-pipeline -> Scan Multibranch Pipeline Log

Project configuration must be done in the project pipeline or Jenkinsfile (NOT in the branch)

3.2.3. Adjust and play Jenkinsfile directly in the version with Replay

Dashboard->my-pieline->branch->Build History(choose last)-> Replay -> Main Script-> Change -> Run

3.2.4. Add Groovy in the Jenkinsfile with the script

...
            steps {
                echo 'building the application...'
                script{
                    def test = 2+2 > 3 ? 'cool' : 'not good'
                    echo test
                }
            }
...

3.2.4. Restart from Stage

Dashboard->my-pieline->branch->Build History(choose dropdown menu) -> Restart from Stage -> Chose Stage -> Run

4. Trigger Jenkins build automatically

Git Integration
-Push Notification
-Polling for Changes
commit -> Git Repo -> trigger build

2 ways to trigger a build -Push Notification: Version Control notifies Jenkins on a new commit(more efficient)
-Polling: Jenkins polls in regular intervals

4.1. Push notification

Trigger build:
-Install the Jenkins plugin based on your Version Control System
-Configure Repository Server Hostname
-Access Token and Credential in Dashboard->Manage Jenkins->Configure System->GitHub->Add GitHub Server
Developer->commits->Git Repo->pushes to->Jenkins Webhook url->Jenkins lets me trigger that build for you

4.2. Polling

Dashboard->my-pipeline->Configuration->Scan Multibranch Pipeline Triggers->Interval(1 hour)->Save Goof strategy to configure both as a backup plan

5. Jenkinsfile configuration

Scripted pipeline. Pipeline as a code -scripted pipeline -is created in your repository

5.1. From Scripted to Declarative Pipeline Syntax

Pipeline Syntax

  • Scripted -first syntax
    -Groovy engine
    -Advanced scripting capabilities, high flexibility
    -difficult to start
  • Declarative -recent addition -easier to get started, but not that powerful -pre-defined structure

5.2. Required Fields of Jenkinsfile

-'pipeline' must be top-level -'agent' -where to execute -'stages' -where the 'work' happens -'stage' and 'steps' -'step' -where execute a command in the Jenkins server

pipeline {
    agent any
    stages {
        stage("build") {
            steps {
                echo 'building the application...'
            }
        }
        stage("test") {
            steps {
                echo 'testing the application...'
            }
        }
        stage("deploy") {
            steps {
                echo 'deploying the application...'
            }
        }
    }
}

5.3. Post Build Actions in Jenkinsfile

Execute some logic AFTER all stages executed Conditions:
-always -success -failure Build Status or Build Status Changes

pipeline {
    agent any
    stages {
...        
    }
    post {
        always{}
        success{}
        failure{}
    }
}

5.4. Define Conditionals / When expression

The "When condition" directs if steps will be executed

...
        stage("build") {
            when {
                expression {
                    BRANCH_NAME == 'dev' && CODE_CHANGE == true
                }
            }
            steps {
                echo 'testing the application...'
            }
        }
        stage("test") {
            when {
                expression {
                    BRANCH_NAME == 'dev' || BRANCH_NAME == 'master'
                }
            }
            steps {
                echo 'testing the application...'
            }
        }
...

5.5. Environmental Variables

Jenkins variable located here http://localhost:8080/env-vars.html/ Double quotes must use.

pipeline {
    agent any
    environment {
        NEW_VERSION = '1.3.0'
    }
    stages {
        stage("build") {
            steps {
                echo "building the application version ${NEW_VERSION}"
            }
...

5.5.1. Using Credentials in Jenkinsfile

  1. Define Credentials in Jenkins GUI (Global Domain)
  2. "credentials("credentialId")" binds the credentials to your env variable
  3. For that you need a "Credentials Binding" Plugin
pipeline {
    agent any
    environment {
        NEW_VERSION = '1.3.0'
        SERVER_CREDENTIAS = credentials('server-credentials') // credentialID from Jenkins server 
    }
....

5.5.2. Credentials in line

...
stage("deploy") {
    steps {
        echo 'deploying the application...'
        withCredentials([
            usernamePassord(credentials: 'server-credentials', usernameVariable: USER, passwordVariable: PWD)
        ]){
            sh "some script ${USER} ${PWD}" 
        }
    }
}
...

5.5.3. Credentials Plugins:

-Credentials -Credentials Binding

5.6. Tools attribute for building tools

Access build tools for your projects
Only 3 build tools are available: gradle, maven, jdk the tool should be preinstalled in the Jenkins server Dashboard -> Manage Jankins -> Global Tool Configuration

pipeline {
    agent any
    tools{
        maven 'Maven'
    }
...

5.7. Jenkins Parameters

Types of Parameter:
-string (name, defaultValue, description) -choice (name, choices, description) -booleanParam (name, defaultValue, description)

pipeline {
    agent any
    parameters {
        string(name: 'VERSION', defaultValue: '', description: 'version to deploy on prod')
        choice(name: 'VERSION', choices: ['1.1.0', '1.2.0', '1.3.0'])
        booleanParam(name: 'executeTests', defaultValue: true)
    }
...

            when {
                expression {
                    params.executeTests == true
                }
            }
...

5.7.1. Build with Parameters

If there is an empty parameter or choice, go to "Build with Parameters"
Jenkins Dashboard -> my-pipeline -> master-> Build with Parameters

5.8. Groovy script

All environmental variables in Jenkinsfile are available in the groovy script

5.8.1. Groovy file
def buildApp() {
    echo 'building the application from grovy script'
}
def testApp() {
    echo 'testing the application from grovy script'
}
def deployApp() {
    echo 'deploying the application from grovy script'
}
return this
5.8.2. Groovy script in Jenkinsfile
def gv
pipeline {
    agent any
    parameters {
        string(name: 'VERSION', defaultValue: '', description: 'version to deploy on prod')
        choice(name: 'VERSION1', choices: ['1.1.0', '1.2.0', '1.3.0'])
        booleanParam(name: 'executeTests', defaultValue: true)
    }
    environment {
        NEW_VERSION = '1.3.0'
        SERVER_CREDENTIAS = credentials('server-credentials') // credentialID from jendins server 
    }
    stages {
        stage("init") {
            steps {
                script {
                    gv = load "script.groovy"
                }
            }
        }
        stage("build") {
            when {
                expression {
                    BRANCH_NAME == 'dev' && CODE_CHANGE == true
                }
            }
            steps {
                echo "building the application version ${NEW_VERSION}"
                script {
                    gv.buildApp()
                }
            }
        }

5.9. Adjust and play Jenkinsfile directly in the version with Replay

Dashboard->my-pieline->branch->Build History(choose last)-> Replay -> Main Script-> Change -> Run

Learn Jenkins! Complete Jenkins Course - Zero to Hero https://youtu.be/6YZvp2GwT0A

1. Jenkins Infrastructure - Master Server and Agents

Master Server -Controls Pipelines -Schedules Builds Agents/Minions -Perform the Build

Workflow:

  1. Commit Triggers Pipeline
  2. Agent Selected based on Configured Labels
  3. Agent Runs Build

Agent Types:

  • Permanent Agents: Dedicated Servers for Running jobs
  • Cloud Agents: Ephemeral/Dynamic Agents spun up on demand (Docker, Kubernetes)

Build Types:

  • Freestyle Build
    • Simplest method to create a build
    • "Feels" like Shell Scripting
  • Pipelines
    • Use theGroovy Syntax
    • Use Stages to break down Components of builds (clone, build, test, package, deploy)

2. Setting up Jenkins Docker Container

2.0. Create or clone the project

git clone https://github.com/git_user_name/your_repo cd your_repo

2.1. Jenkins BlueOcean Docker Image

Blue Ocean allows users to graphically create, visualize and diagnose CI/CD Pipelines

2.1.1. Dockerfile

FROM jenkins/jenkins:2.332.3-jdk11
USER root
RUN apt-get update && apt-get install -y lsb-release
RUN curl -fsSLo /usr/share/keyrings/docker-archive-keyring.asc \
  https://download.docker.com/linux/debian/gpg
RUN echo "deb [arch=$(dpkg --print-architecture) \
  signed-by=/usr/share/keyrings/docker-archive-keyring.asc] \
  https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list
RUN apt-get update && apt-get install -y docker-ce-cli
USER jenkins
RUN jenkins-plugin-cli --plugins "blueocean:1.25.3 docker-workflow:1.28"

2.1.2. Build Jenkins BlueOcean Docker Image

docker build -t myjenkins-blueocean:2.332.3-1 .

2.2. Create network 'jenkins'

docker network create jenkins

2.2.1 Verify docker networks

`docker network ls'

2.3. Run Docker Container

2.3.1. MacOS / Linux

docker run --name jenkins-blueocean --restart=on-failure --detach \
  --network jenkins --env DOCKER_HOST=tcp://docker:2376 \
  --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 \
  --publish 8080:8080 --publish 50000:50000 \
  --volume jenkins-data:/var/jenkins_home \
  --volume jenkins-docker-certs:/certs/client:ro \
  myjenkins-blueocean:2.332.3-1

2.3.2. Windows

docker run --name jenkins-blueocean --restart=on-failure --detach `
  --network jenkins --env DOCKER_HOST=tcp://docker:2376 `
  --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 `
  --volume jenkins-data:/var/jenkins_home `
  --volume jenkins-docker-certs:/certs/client:ro `
  --publish 8080:8080 --publish 50000:50000 myjenkins-blueocean:2.332.3-1

2.3.1 Verify Running Docker Container

docker ps

2.4. Get Password

docker exec jenkins-blueocean cat /var/jenkins_home/secrets/initialAdminPassword

and put in the password field in localhost:8080

2.5. Connect to Jenkins

https://localhost:8080/

2.5.1. Install suggested plugins

2.5.2. Create the First Admin User

Illya 1234 Illya K my@email.com Jenkins URL http://localhost:8080/ Save and Finish

2.6 Create New Item

Freestyle project name my_first_job

2.6.1. Simple Build

Dashboard->my_first_job->Configure->General->Build->Add build step->Execute shell echo "Hello world!!!" ->Save ->Build Now Output:

  • echo Hello world!!! // command Hello world!!! // result

2.6.2. List of available environment variables

Dashboard->my_first_job->Configure->General->Build->See the list of available environment variables echo "The build ID of this job is ${BUILD_ID}" echo "The built URL is ${BUILD_URL}" ->Save ->Build Now

2.6.3. Create the file

Dashboard->my_first_job->Configure->General->Build->Add build step->Execute shell ls -ltr echo "1234" > test.txt ls -ltr ->Save ->Build Now There is a file in Dashboard->my_first_job->Workspace

2.6.3. Delete Workspace

Dashboard->my_first_job->Configure->Build Triggers-> Build Environment-> Delete workspace before build starts ->Save ->Build Now

2.x. Installation Reference:

https://www.jenkins.io/doc/book/installing/docker/

2.x+1. alpine/socat container to forward traffic from Jenkins to Docker Desktop on Host Machine

https://stackoverflow.com/questions/47709208/how-to-find-docker-host-uri-to-be-used-in-jenkins-docker-plugin

docker run -d --restart=always -p 127.0.0.1:2376:2375 --network jenkins -v /var/run/docker.sock:/var/run/docker.sock alpine/socat tcp-listen:2375,fork,reuseaddr unix-connect:/var/run/docker.sock
docker inspect <container_id> | grep IPAddress

2.x+2. Using my Jenkins Python Agent

docker pull devopsjourney1/myjenkinsagents:python

Clone this wiki locally