Cisco Intersight Infrastructure-as-Code Demo

Disclaimer

This is NOT an official Cisco application and comes with absolute NO WARRANTY!
Please check the LICENSE for further information.

Credits
This demo has been largely inspired by the great work done by Ned Bellavance for his 'Getting Started' and 'Deep Dive' Terraform courses on Pluralsight. Although it has been mostly reworked and extended to work with Intersight and Vault, some constructs and strategies have been reused in this repository.

Introduction

The scope of this project is to demonstrate how Cisco Intersight can fit an existing CI/CD environment with centralized secret management and multiple teams sharing resources.
This repository contains the code and the instructions to setup the whole demo on your laptop.

At the end of the setup you will have:

• Two GitHub private repositories, one for the Compute team, one for the Security Team
• A Jenkins instance, preconfigured with two pipelines and the necessary credentials and plugins
• A Vault instance configured to authenticate on behalf of Consul. This Vault instance will also store your Intersight credentials
• A Consul instance that stores both compute and security configuration parameters and remote state

Notes

• Once the setup is complete, you can "pause" it to free resources with minikube stop
  
• You can resume the demo setup with minikube start. However, you are required to recreate the port-forwarding rules as well as Vault configuration. Since we are running in dev mode, once the pod restarts, it losts all data. (see below to recreate the port-forwarding rules and Vault configuration)
  
• The first time you run the pipelines, it may take up to 20 minutes as Jenkins will need to download the container agent images for the first time. Subsequent runs should happen much faster. The same is true with the installation of Vault, Consul and Jenkins as minikube will need to download the images first

Usage

Once the demo environment is up and running, connect to Jenkins and execute the pipelines to deploy the demo code to Intersight.
The security-team pipeline can only be executed after the compute-team pipeline, as it will leverage its remote Terraform state file.

The demo environment supports whetever Terraform code you have in your SCM repository, you are definitely encouraged to fork this repo and create additional use cases!

Prerequisites

• Docker Desktop - Ensure you reserve at least 4GB Memory for Docker Desktop
• minikube
• helm
• awk
• git
• terraform
• consul client
• vault client
• An Intersight secret file (get it from intersight.com, settings -> API keys )
• A GitHub personal access token

Installation

Warning: This demo will create policies and a server profile. This profile will be associated to an existing server.
In Intersight, identify a server you can use and take note of its management IP.

Warning: READ before you copy/paste! Some steps require manual population of environment variables based on your setup

From a terminal, clone this repository

git clone https://github.com/rtortori/iatc-intersight-iac.git

Move to the repo directory and delete the existing repository

cd iatc-intersight-iac && rm -rf .git

Copy your Intersight Secret in the demo-setup/intersight directory (replace "/path/to" with the actual directory)

cp /path/to/SecretKey.txt demo-setup/intersight/SecretKey.txt

In the 'team-compute/2_main/3_datasources.tf' file, replace "192.168.30.22" the the management IP of the server you want to associate the server profile to

export SERVER_MGT_IP="server.mgt.ip.address"
sed -i '' -e "s/192.168.30.22/$SERVER_MGT_IP/g" team-compute/2_main/3_datasources.tf

Set the Intersight API key as an environment variable

export INTERSIGHT_API_TOKEN="your_token_here"

Set your GitHub token and username as an environment variable
(E.g. GITHUB_USERNAME="rtortori")

export GITHUB_TOKEN="your_token_here"
export GITHUB_USERNAME="myuser"

Start minikube

minikube start

Deploy Consul

helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm install -f ./demo-setup/consul/consul_values.yaml consul hashicorp/consul --version 0.29.0

Deploy Jenkins

helm repo add jenkins https://charts.jenkins.io
helm repo update
helm install -f ./demo-setup/jenkins/jenkins_values.yaml jenkins jenkins/jenkins --version 3.1.2

Deploy Vault

helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault --version 0.8.0 \
    --set "server.dev.enabled=true" \
    --set "injector.enabled=false" \
    --set "ui.enabled=true"

Ensure all pods are in the 'Running' state. They also need to show '1/1'
If some pods are not ready yet, just wait a few more minutes and retry the command. It may take some time depending on your internet connection

kubectl get pods

Create the port-forwarding rules to access Jenkins, Vault and Consul

kubectl port-forward service/consul-consul-ui 8500:80 --address 0.0.0.0 >/dev/null&
kubectl port-forward service/jenkins 8080:8080 >/dev/null&
kubectl port-forward service/vault-ui 8200:8200 >/dev/null&

Set Environment variables to access deployed resources (Vault token will be 'root' as we run it in Dev mode)

export CONSUL_HTTP_TOKEN=$(kubectl get secret consul-consul-bootstrap-acl-token -o=jsonpath={.data.token} | base64 -d)
export JENKINS_URL="http://127.0.0.1:8080"
export JENKINS_USERNAME="admin"
export JENKINS_PASSWORD="$(kubectl exec --namespace default -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/chart-admin-password && echo)"
export COOKIE_JAR=/tmp/cookies
export VAULT_TOKEN=root
export VAULT_ADDR='http://0.0.0.0:8200'

Consul Setup

cd demo-setup/consul
terraform init
terraform plan -out consul.tfplan
terraform apply --auto-approve consul.tfplan

Create Intersight Compute and Security Configuration in Consul

consul kv put intersight_compute/configuration/compute_config @compute_config.json
consul kv put intersight_security/configuration/security_config @security_config.json

Create an ACL token for Vault to manage Consul Authentication

export GLOB_MGT_TOKEN=$(consul acl token create -policy-name=global-management | grep SecretID | awk '{print $2}')

Enable the consul secret engine in Vault

vault secrets enable consul

Configure Vault to authenticate to Consul

vault write consul/config/access \
    address=consul-consul-server:8500 \
    token=$GLOB_MGT_TOKEN

Configure compute and security roles in Vault and map to Consul Policies

vault write consul/roles/intersight_compute policies=intersight_compute
vault write consul/roles/intersight_security policies=intersight_security

Change Directory to the intersight directory

cd ../intersight

Write Intersight secrets to Vault

vault secrets enable -path=intersight kv
vault kv put intersight/credentials/cert cert=@SecretKey.txt
vault kv put intersight/credentials/api_key api_key=$INTERSIGHT_API_TOKEN

Configure credentials in Jenkins. Set Jenkins crumbs

JENKINS_CRUMB=$(curl --silent --cookie-jar $COOKIE_JAR $JENKINS_URL'/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)' -u $JENKINS_USERNAME:$JENKINS_PASSWORD)

Set GitHub and Vault Credentials in Jenkins.
For this demo we will use the same token, but in production you will have policies and different tokens

# GH credentials
curl -X POST "http://$JENKINS_USERNAME:$JENKINS_PASSWORD@127.0.0.1:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR  -H $JENKINS_CRUMB \
--data-urlencode 'json={
  "": "0",
  "credentials": {
    "scope": "GLOBAL",
    "id": "GitHub_Credentials",
    "username": "dummy",
    "password": "'"$GITHUB_TOKEN"'",
    "description": "GH Credentials",
    "$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
  }
}'

# Vault Compute Credentials
curl -X POST "http://$JENKINS_USERNAME:$JENKINS_PASSWORD@127.0.0.1:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR  -H $JENKINS_CRUMB \
--data-urlencode 'json={
 "": "0",
  "credentials": { 
  "scope": "GLOBAL", 
  "id": "Intersight_Compute_Vault", 
  "secret": "root", 
  "description": "Vault token for Compute team", 
  "$class": "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl" 
 }
}' 

# Vault Security Credentials
curl -X POST "http://$JENKINS_USERNAME:$JENKINS_PASSWORD@127.0.0.1:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR  -H $JENKINS_CRUMB \
--data-urlencode 'json={
 "": "0",
  "credentials": { 
  "scope": "GLOBAL", 
  "id": "Intersight_Security_Vault", 
  "secret": "root", 
  "description": "Vault token for Security team", 
  "$class": "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl" 
 }
}'

You will now need to login to Jenkins. Paste the following command and connect with your browser

echo "Login to Jenkins at $JENKINS_URL\nUsername: $JENKINS_USERNAME \nPassword: $JENKINS_PASSWORD"

Once connected, go to:

• "Manage Jenkins" -> "Global Tool Configuration" -> "Terraform" -> "Add Terraform" -> Set "Name" = terraform
• "Install from bintray.com" -> Select "Terraform 0.13.5 linux (amd64)" -> Hit "Save"

Switch back to your terminal and move to the github directory

cd ../github

Configure your GitHub token in the terraform configuration file

sed -i '' -e "s/my_gh_token/$GITHUB_TOKEN/g" 2_variables.tf

Create the GitHub repositories and the Jenkins pipeline

terraform init
terraform plan
terraform apply -auto-approve

Move to the team-compute directory

cd ../../team-compute

Init the Compute repository and push to GitHub

git init
echo ".terraform/" >> .git/info/exclude
echo "*.tfstate" >> .git/info/exclude
echo "*.tfstate.backup" >> .git/info/exclude
git add .
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/$GITHUB_USERNAME/intersight_iac_demo_compute.git
git push -u origin main

Move to the team-security directory

cd ../team-security

Init the Security repository and push to GitHub

git init
echo ".terraform/" >> .git/info/exclude
echo "*.tfstate" >> .git/info/exclude
echo "*.tfstate.backup" >> .git/info/exclude
git add .
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/$GITHUB_USERNAME/intersight_iac_demo_security.git
git push -u origin main

Configuration

The Jenkinsfile(s) in both pipelines include most of the variables used by Terraform.
You may want to customize the following - or not :)

• TF_VAR_resource_prefix: Each resource created in Intersight will have this prefix
• TV_VAR_organization: The name of the Intersight organization you want your resources in. Most likely is going to be 'default' but in some cases you may want to use an alternative organization

Clean-up

To clean-up objects created in Intersight, just run the security and the compute pipelines (in this order) and when it asks for confirmation, click on 'Abort'. It will execute a terraform destroy -auto-approve and everything will disappear.

To clean-up the GitHub repositories and delete all traces of this demo

cd demo-setup/github && terraform destroy -auto-approve

minikube stop 
minikube delete

unset INTERSIGHT_API_TOKEN
unset GITHUB_TOKEN
unset GITHUB_USERNAME
unset CONSUL_HTTP_TOKEN
unset JENKINS_USERNAME
unset JENKINS_PASSWORD
unset JENKINS_CRUMB
unset COOKIE_JAR
unset VAULT_TOKEN
unset VAULT_ADDR
unset GLOB_MGT_TOKEN