Skip to content

This has code for installing jenkins on linux machine and creating an AMI

Notifications You must be signed in to change notification settings

RamaRaju-personal-org/ami-jenkins

Repository files navigation

Configure your domain by creating a AWS Route53 Public Hosted Zone

AWS Route53 Subdomain Delegation

Jenkins AMI with Hashicorp Packer

📦 Packer

Building custom AWS AMI using Packer from HashiCorp.

⤵️ Installing Packer

Install Packer using Homebrew (only on MacOS):

For any other distros, please follow the setup guide in the official docs.

  • First, install the HashiCorp tap, a repository of all our Homebrew packages:
brew tap hashicorp/tap
  • Now, install Packer with hashicorp/tap/packer:
brew install hashicorp/tap/packer
  • To update to the latest, run:
brew upgrade hashicorp/tap/packer
  • After installing Packer, verify the installation worked by opening a new command prompt or console, and checking that packer is available:
packer

Note

If you get an error that packer could not be found, then your PATH environment variable was not set up properly. Please go back and ensure that your PATH variable contains the directory which has Packer installed. Otherwise, Packer is installed and you're ready to go!

🔧 Building Custom AMI using Packer

Packer uses Hashicorp Configuration Language(HCL) to create a build template. We'll use the Packer docs to create the build template file.

Note

The file should end with the .pkr.hcl extension to be parsed using the HCL2 format.

Create the .pkr.hcl template

The custom AMI should have the following features:

Note

The builder to be used is amazon-ebs.

  • OS: Ubuntu 22.04 LTS
  • Build: built on the default VPC
  • Device Name: /dev/sda1/
  • Volume Size: 8GiB
  • Volume Type: gp2
  • Have valid provisioners
  • Pre-installed dependencies using a shell script.
  • Jenkins pre-installed on the AMI.

Shell Provisioners

This will automate the process of updating the OS packages and installing software on the AMI and will have our application in a running state whenever the custom AMI is used to launch an EC2 instance. It should also copy artifacts to the AMI in order to get the application running. It is important to bootstrap our application here, instead of manually SSH-ing into the AMI instance.

Install application prerequisites, middlewares and runtime dependencies here. Update the permission and file ownership on the copied application artifacts.

Note

The file provisioners must copy the application artifacts and configuration to the right location.

Custom AMI creation

To create the custom AMI from the .pkr.hcl template created earlier, use the commands given below:

  • If you're using Packer plugins , run the init command first:
# Installs all packer plugins mentioned in the config template
packer init .
  • To format the template, use:
packer fmt .
  • To validate the template, use:
# to validate syntax only
packer validate -syntax-only .
# to validate the template as a whole
packer validate -evaluate-datasources .
  • To build the custom AMI using packer, use:
packer build <filename>.pkr.hcl

Packer HCL Variables

To prevent pushing sensitive details to your version control, we can have variables in the <file-name>.pkr.hcl file, and then declare the actual values for these variables in another HCL file with the extension .pkrvars.hcl.

If you want to validate your build configuration, you can use the following command:

packer validate -evaluate-datasources --var-file=<variables-file>.pkrvars.hcl <build-config>.pkr.hcl

Note

To use the -evaluate-datasources parameter, you'll have to update packer to v1.8.5 or greater. For more details, refer this issue.

To use this variables files when creating a golden image, use the build command as shown:

packer build --var-file=<variables-file>.pkrvars.hcl <build-config>.pkr.hcl

Note

It is considered best practice to build a custom AMI with variables using HCP Packer!

⤵️ Install required software

In order for Jenkins to run it requires Java

☕️ Java Installation

# Installing Java
sudo apt update --quiet
sudo apt install openjdk-11-jdk -y
sudo apt update --quiet
# Validate installation
java -version

Jenkins Installation

# Installing Jenkins

sudo apt update --quiet
# Debian package repository of Jenkins to automate installation and upgrade.
# To use this repository, first add the key to the system:
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc >/dev/null

# Add a Jenkins apt repository entry:
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list >/dev/null

# update local package index and install Jenkins
sudo apt update --quiet
sudo apt install jenkins -y
# check the status of Jenkins
sudo systemctl status jenkins

📦 NodeJS Installation

# https://github.com/nodesource/distributions#installation-instructions
# Download and import the Nodesource GPG key
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo \
  gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg

# Create a deb repository
NODE_MAJOR=20
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo \
  tee /etc/apt/sources.list.d/nodesource.list

# Run update and install
sudo apt-get update && sudo apt-get install nodejs -y

# Check Node version:
echo "Node $(node --version)"

🐳 Docker Installation

# Add Docker's official GPG key:
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" |
  sudo tee /etc/apt/sources.list.d/docker.list >/dev/null

# Install Docker:
sudo apt-get update && sudo apt-get install docker-ce -y

# Provide relevant permissions
sudo chmod 666 /var/run/docker.sock
sudo usermod -a -G docker jenkins

# Check Docker version
echo "Docker $(docker --version)"

🔒 Configure Caddy Service

# Install and configure keyring for caddy stable release:
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo \
  gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee \
  /etc/apt/sources.list.d/caddy-stable.list

# Install caddy:
sudo apt update # required to refresh apt with the newly installed keys
sudo apt install caddy -y

To configure reverse-proxy with Caddy, refer the official documentation here

Example

vi /etc/caddy/Caddyfile

# inside Caddyfile
----------------------------------------------------------------
{
    acme_ca https://acme-staging-v02.api.letsencrypt.org/directory # only in case of stagging
} 
cicd.ramaraju.cloud {
  reverse_proxy http://127.0.0.1:8080
}
-----------------------------------------------------------------

Note

To remove reverse proxy error on Jenkins server: Jenkins->Manage->Configure->Jenkins URL->set it to "caddy1".

⚙️ Configuring Jenkins

⬇️ Installing plugins

We will configure the Jenkins server using the install.sh script that configures and installs plugins for us in an automated fashion. There are a couple of plugins that will help us setup the Jenkins server with Jenkins Configuration as Code:

  • job-dsl: Configure seed jobs to setup multi-branch pipelines
  • configuration-as-code: Configure Jenkins with a JCasC yaml file that installs required tools and creates users
  • configuration-as-code-groovy: Configure Jenkins with a JCasC yaml file that runs the Groovy scripts defined in the seed jobs

In order to install the plugins on the EC2 instance, we need to download and run the plugin-installation-manager-tool from GitHub.

# Install Jenkins plugin manager tool:
wget --quiet \
  https://github.com/jenkinsci/plugin-installation-manager-tool/releases/download/2.12.13/jenkins-plugin-manager-2.12.13.jar

Next, we need to install the list of plugins mentioned in the plugins.txt file:

The plugins.txt file contains the names and the versions of the plugins that we would need to configure Jenkins and run CI/CD jobs.

# Install plugins with jenkins-plugin-manager tool:
sudo java -jar ./jenkins-plugin-manager-2.12.13.jar --war /usr/share/java/jenkins.war \
  --plugin-download-directory /var/lib/jenkins/plugins --plugin-file plugins.txt

Remember that we need to update the user and group permissions to jenkins for these plugins:

# Update users and group permissions to `jenkins` for all installed plugins:
cd /var/lib/jenkins/plugins/ || exit
sudo chown jenkins:jenkins ./*

🧾 JCasC (Jenkins Configuration As Code)

In order to configure Jenkins with Configuration as Code, we need to define a yaml file with some basic fields:

jobs:
  - file: ./<your_seed_job>.groovy
unclassified:
  location:
    url: https://<localhost>:8080

🧳 Seed jobs using Groovy scripts

To setup multi-branch pipelines, we'll use Groovy scripts:

multibranchPipelineJob('job-name') {
  branchSources {
    github {
      id('unique-job-id')
      scanCredentialsId('github-webhook-app-credentials')
      repoOwner('repository-owner')
      repository('repository-name')
    }
  }

  orphanedItemStrategy {
    discardOldItems {
      numToKeep(-1)
      daysToKeep(-1)
    }
  }
}

We have to update the user and group permissions for the JCasC and groovy files:

# Update file ownership
cd /var/lib/jenkins/ref/ || exit
sudo chown jenkins:jenkins <your-jcasc>.yaml <your_seed_job>.groovy

⏫ Update jenkins service

To disable the initial Jenkins setup wizard and to configure the Jenkins server using the JCasC file, we'll need to update the jenkins.service systemd service file:

# Configure JAVA_OPTS to disable setup wizard
sudo mkdir -p /etc/systemd/system/jenkins.service.d/
{
  echo "[Service]"
  echo "Environment=\"JAVA_OPTS=-Djava.awt.headless=true -Djenkins.install.runSetupWizard=false -Dcasc.jenkins.config=/var/lib/jenkins/ref/jcasc.yaml\""
} | sudo tee /etc/systemd/system/jenkins.service.d/override.conf

Finally, restart your jenkins service:

# restart jenkins service
sudo systemctl daemon-reload
sudo systemctl stop jenkins
sudo systemctl start jenkins