Skip to content

DevOps

Jump to bottom
ILLYAKO edited this page Dec 7, 2022 · 72 revisions

DevOps foundation

Docker, Docker Compose

Test (Unit test and Integration test)

RSpec, Capybara, Selenium

Deploy

Terraform

CI/CD Pipeline (build, test, deploy)

Jenkins

1. Docker

Docker is a platform for running applications and their dependencies in isolated environments called containers, on nearly any operating system. Homebrew (brew.sh)for installing applications on Mac(Linux) Aptitude, YUM, and YaST are package managers that let you install anything on the Mac

1.1. Docker Installation

1.1.1. Install Homebrew on Linux

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

1.1.2. Install Docker on Linux

$ brew install cask docker - cask(third-part repository) installs Docker, Docker CLI, Docker Compose

1.1.3. Install Docker on Windows

1.1.4. Run Docker Desktop (Windows)

1.1.5. Test Docker

$ docker run hello-world

1.2. Dockerfile

The Dockerfile is a manifest that describes the image that the container will use when we run it. All of the configuration dependencies and environment dependencies and everything that the application needs will be expressed within the Dockerfile.
Docker will do a few things.

  1. The Docker reads and parses the Dockerfile.
  2. The Docker fetches the parent image that this image is going to use. If there is no parent image that you're going to use, you would start from scratch.
  3. The Docker runs any commands, within the Dockerfile that is on top of that image, lastly if defined you can set a process that runs whenever a container from that image is spun up.

1.2.1. Docker Hub is a repository of docker images.

The 'alpine' is the smallest version of the image

1.2.2. Dockerfiles lines:

FROM nginx:alpine                                   #parent image:tag

MAINTAINER Illya Korotun <illya.korotun@gmail.com>  #author
COPY website /website                               #copy context into the image host directory to the container directory
COPY nginx.conf /etc/nginx/nginx.cong               #copy the config file to the container

EXPLOSE 80 #we want to map one(80) of the ports on my Mac to the port on the container.

1.2.3. Build docker image

$ docker build --tag website . # create an image with tag/name "website" from "." (current directory)

1.2.4. Run Container

$ docker run --publish 80:80 website # publish on ports HOST:Container

1.3. Docker Compose manifest

Docker compose is a simple and lightweight platform for running multiple container applications in a single stack and creating a network.

1.3.1 docker-compose.yml
version: '3.7'   # docker compose version
services:
  website:       # service name usually equals the container name
    build:       # look for Dockerfile and build an image
      context: . # Dockerfile address "."-current directory
    ports:       # container ports HOST:Container
      - 80:80    # Dockerfile address "."-current directory
1.3.2 Run docker-comose and build container

$ docker-compose run website

1.3.3 Up docker-comose and build container and make port mapping

$ docker-compose up

1.3.4 Show a list of the docker-compose container running in

$ docker-compose ps

1.3.5 Show a list of the docker container running

$ docker ps

1.3.6 Stop the running container and remove the network

$ docker-compose down

2. Testing App with RSpec, Capybara, and Selenium

  • RSpec is a Ruby-based testing framework. RSpec is looking for a folder called Spes and _spec.rb files (ex. page_spec.rb).
  • Capybara is a tool that lets you use a web driver to create a browser and interact with the website. It gets the HTML of a website, run Javascript, and tests things like a button looking like it should look.
  • The web driver, which is what spins up the browser and runs tests against it.
  • Rack Test is a very simple web browser that just renders HTML and tests it, but it can't do more complicated things like running Javascript or computing CSS.
  • Selenium is a web driver that spins up a real web browser and tests against whatever the web browser sees, which is nice because our website does have computed CSS, and spinning up a real web browser and getting the results of that computed CSS.

2.1. RSpec

2.1.1. Create new directories 'spec' and 'spec/unit' in the project root directory

$ mkdir spec
$ mkdir spec/unit

2.1.2. Create a new file page_spec.rb

$ touch spec/unit/page_spec.rb

# spec/unit/page_spec.rb
require 'capybara'
require 'capybara/dsl'

describe "Example page render unit tests" do
  it "Should show the Explore California logo" do
  end
end

2.1.3 Edit docker-compose adding the 'unit-test'

version: "3.7"
services:
  website:
    build:
      context: .
    ports:
      - 80:80
  unit-test:                       # new service
    volumes:                       # mount the existing volume to the container
      - "$PWD:/app"                #current directory to /app
    build:                         # build a new image
      context: .                   #current context
      dockerfile: rspec.dockerfile #spesific dockerfile
    command:
      - --pattern                  # entrypoint option
      - /app/spec/unit/*_spec.rb        # entrypoint option test target

2.1.4 Create new file rspec.dockerfile

$ touch rspec.dockerfile

# create from base ruby image
FROM ruby:alpine 
MAINTAINER Illy Korotun <illya.korotun@gmail.com>

# apk is pacage from alpine
# add package ruby-nokogiri for parsing 
RUN apk add build-base ruby-nokogiri

# install in the container from the ruby gem library rubygems.org
RUN gem install rspec capybara selenium-webdriver

#Which process should start when the container is starting. Use double equate 
ENTRYPOINT ["rspec"]

Two way say container how to run CMD and ENTRYPOINT
Every container start with the default command #/bin/sh -c
CMD ["rspec"] # is mean #/bin/sh -c "rspec"

ENTRYPOINT is running the command where it placed
ENTRYPOINT ['option1', 'option2'] # is mean docker run this_image --option1 --option2
ENTRYPOINT ['rspec'] # is mean docker run this_image rspec

2.1.5 Execute docker-compose -d to hide the output of the container

$ docker-compose up -d website

2.1.6 Run container unit-test with the option to remove the container after use

$ docker-compose run --rm unit-test
The dot '.' should be shown as a result of the successful test

2.1.7 Docker Compose config file

$ docker-compose config

2.1.8 Run container 'website' in the background

$ docker-compose up -d website

2.1.9 Add tests

# spec/unit/page_spec.rb
require 'capybara'
require 'capybara/dsl'
require 'selenium-webdriver'

include Capybara::DSL
Capybara.app_host = "http://website" # Using Selenium; connect over network
Capybara.run_server = false # Disable Rack since we are using Selenium.
Capybara.register_driver :selenium do |app|
  Capybara::Selenium::Driver.new(
    app,
    browser: :remote,
    url: "http://#{ENV['SELENIUM_HOST']}:#{ENV['SELENIUM_PORT']}/wd/hub",
    capabilities: Selenium::WebDriver::Remote::Capabilities.chrome( # desired_capabilities was replaced by capabilities in new Capybara 4
      "chromeOptions" => {
        "args" => ['--no-default-browser-check']
      }
    )
  )
end
Capybara.default_driver = :selenium

describe "Example page render unit tests" do
  it "Shows the Explore California logo" do
    visit('/')
    expect(page.has_selector? '.logo').to be true
  end
end

2.1.10 Add Selenium to docker-compose.yml

version: "3.7"
services:
  selenium:
    image: selenium/standalone-chrome-debug # debug version include VNC
    ports:
      - 4444:4444
      - 5901:5900 # VNC ports. The 5900 is the default VNC port
  unit-test: # new service
    build: # build a new image
      dockerfile: rspec.dockerfile #spesific dockerfile
      context: . #current context
    volumes: # mount the existing volume to the container
      - $PWD:/app #current directory to /app
    command:
      - --pattern # entrypoint option
      - /app/spec/unit/*_spec.rb # entrypoint option test target
  website:
    build:
      context: .
    ports:
      - 80:80

2.1.11 Run service website selenium and Run container unit-tests

$ docker-compose up -d website selenium
$ docker-compose run --rm unit-test

2.2.1 Open VNC (VNC Viewer for Windows)

localhost:5901
password: secret

2.1.13 Run container unit-tests

$ docker-compose run --rm unit-test

3. Infrastructure as Code with Terraform

3.1 Create Terraform Dockerfile

# create from Linux alpine
FROM alpine
MAINTAINER Illya Korotun <illya.korotun@gmainl.com>

#download Terraform from https://developer.hashicorp.com/terraform/downloads

RUN wget -O /tmp/terraform.zip https://releases.hashicorp.com/terraform/0.12.12/terraform_0.12.12_linux_amd64.zip
RUN unzip /tmp/terraform.zip -d /
RUN apk add --no-cache ca-certificates curl

# nobody user has privileges to execute
USER nobody 

# check installed terraform in the container
ENTRYPOINT [ "/terraform" ]

3.2 Build Terraform image

$ docker build -t terraform . -f terraform.Dockerfile

3.3 Run container and remove(--rm) it when it done, using interactive command to use shell(--interactive) and shell session(--tty), and change entrypoint for container (sh terraform)

$ docker run --rm --interactive --tty --entrypoint sh terraform
~$ /terraform #check installed terraform in the container
~$ exit

3.4 Build Terraform docker image with name and source file and temporary working directory - context ($PWS or dot ".")(same directory)

docker build --tag terraform --file terraform.dockerfile .

3.5 Run Terraform docker container and remove it and show the version

docker run --rm terraform -version

3.6 Add Terraform to docker-compose

...
service:
  terraform:
    build:
      context: .
      dockerfile: terraform.dockerfile
...

3.6.1 Run Terraform docker-compose

$ docker-compose run --rm terraform

3.7. AWS

AWS - Amazon Web Service is a cloud provider of resources for Amazon data center Elastic Cloud Compute or EC2 allows the creation of virtual machines Amazon S3 is a simple storage service console.aws.amazon.com the project will be copied into the bucket of the S3

3.8 Terraform (terraform.io) configuration file main.tf is the Terraform entry point.

data "aws_iam_policy_document" "bucket_policy" {
  statement {
    sid = "PublicReadGetObject"
    effect = "Allow"
    actions = [ "s3:GetObject" ]
    principals {
      type = "*"
      identifiers = [ "*" ]
    }
    resources = [ "arn:aws:s3:::explorecalifornia.org/*" ]
  }
}

/* We can access properties from data sources using this format:
   ${data.<data_source_type>.<data_source_name>.<property>.

   In this case, we need the JSON document, which the documentation
   says can be accessed from the .json property. */

resource "aws_s3_bucket" "website" {
  bucket = "explorecalifornia.org"  // The name of the bucket.
  acl    = "public-read"            /* Access control list for the bucket.
                                       Websites need to be publicly-available
                                       to the Internet for website hosting to
                                       work. */
  policy = "${data.aws_iam_policy_document.bucket_policy.json}"
  website {
    index_document = "index.htm"   // The root of the website.
    error_document = "error.htm"   // The page to show when people hit invalid pages.
  }
}

output "website_bucket_url" {
  value = "${aws_s3_bucket.website.website_endpoint}"
}

3.9. Init Terraform with docker-compose

docker-compose run --rm terraform init

Clone this wiki locally