Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Docker build in deployment workflows #135

Merged
merged 5 commits into from
Oct 14, 2021
Merged

Conversation

thekaveman
Copy link
Member

@thekaveman thekaveman commented Oct 9, 2021

What this PR does

  1. Uses the docker/setup-buildx-action to setup a Docker Buildx builder for the job
  2. Uses the docker/build-push-action instead of raw docker commands to build the image
  3. Adds layer caching to Docker builds via docker/build-push-action's built-in type=gha for GitHub Actions

Testing the PR

  1. Forked over at https://github.com/thekaveman/benefits

  2. Set up environment similar to this repository

  3. Made a small modification to workflow to skip the actual deploy

  4. Merged change (thekaveman/benefits#1) to see workflow run and succeed with new build steps (some output copied below):

    Set up Docker Buildx

    Run docker/setup-buildx-action@v1
    Docker info
    Creating a new builder instance
    Booting builder
    Inspect builder
    BuildKit version
       moby/buildkit:buildx-stable-1 => buildkitd github.com/moby/buildkit v0.9.1 966bcf4aa3ef397a9d6025fade18f2d59a5bf29d

    Build, tag, and push to Amazon ECR

    Run docker/build-push-action@v2
    Docker info
      /usr/bin/docker buildx build --tag ***.dkr.ecr.***.amazonaws.com/cal-itp-benefits-client:6432a9d0ccee93521950e5ab4e42f1deb1a752d8 --iidfile /tmp/docker-build-push-oYNvON/iidfile --metadata-file 
      /tmp/docker-build-push-oYNvON/metadata-file --cache-from type=gha,scope=benefits --cache-to type=gha,scope=benefits,mode=max --builder builder-b74c8782-c697-4769-b727-5c4bf83b40fc --push .
    #2 [internal] load .dockerignore
    #2 transferring context: 171B done
    
    ...
    
    #17 [ 1/11] FROM docker.io/library/python:3.9-slim@sha256:8ca34144574637bb24baa62a767d1ef8be821d9ee7e6a16fa3ab628cb7f96c27
    #17 sha256:36629b83aba2df8771daed1573adb172bd6878a96fd9327e4f32f0174aec1c5c 233B / 233B 0.1s done
    #17 sha256:7338abefe51cf89eefc207e03876d5c2183b5281ceb57a9067b3e13629303bab 2.64MB / 2.64MB 0.1s done
    #17 sha256:6ad2a11ca37b37ebd142aaa2d25901a91cee64d326c76c0133e42d864a680ed5 1.08MB / 1.08MB 0.1s done
    #17 sha256:e9edbe81a0012ed0e8f2e0a8fba50eb92d4c7b8c3d66a2695c6d6c70457d995e 11.02MB / 11.02MB 0.2s done
    #17 sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa 18.87MB / 31.36MB 0.2s
    #17 sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa 31.36MB / 31.36MB 0.3s
    #17 sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa 31.36MB / 31.36MB 0.4s done
    #17 extracting sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa
    #17 extracting sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa 1.0s done
    #17 extracting sha256:6ad2a11ca37b37ebd142aaa2d25901a91cee64d326c76c0133e42d864a680ed5 0.1s done
    #17 DONE 1.5s
    
    ...
    
    #6 [ 2/11] RUN useradd --create-home --shell /bin/bash calitp &&
      apt-get update &&
      apt-get install -qq --no-install-recommends gettext nginx &&
      mkdir -p /var/cache/nginx &&
      chown -R calitp /var/cache/nginx &&
      mkdir -p /var/lib/nginx &&
      chown -R calitp /var/lib/nginx &&
      mkdir -p /var/log/nginx &&
      chown -R calitp /var/log/nginx &&
      touch /var/run/nginx.pid &&
      chown -R calitp /var/run/nginx.pid &&
      mkdir -p /home/calitp/app/config &&
      mkdir -p /home/calitp/app/run &&
      mkdir -p /home/calitp/app/static &&
      chown -R calitp /home/calitp
    
    #6 0.564 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
    #6 0.575 Get:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
    #6 0.598 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
    #6 0.691 Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [48.0 kB]
    #6 0.781 Get:5 http://deb.debian.org/debian bullseye/main amd64 Packages [8180 kB]
    #6 0.887 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2300 B]
    #6 1.825 Fetched 8430 kB in 1s (6525 kB/s)
    
    ...
  5. Pushed one more small change to trigger the workflow one more time with layer caching! (some output copied below):

    Build, tag, and push to Amazon ECR

    #17 [ 1/11] FROM docker.io/library/python:3.9-slim@sha256:8ca34144574637bb24baa62a767d1ef8be821d9ee7e6a16fa3ab628cb7f96c27
    #17 resolve docker.io/library/python:3.9-slim@sha256:8ca34144574637bb24baa62a767d1ef8be821d9ee7e6a16fa3ab628cb7f96c27 done
    #17 DONE 0.0s
    
    #5 importing cache manifest from gha:10117132071653542490
    #5 DONE 0.5s
    
    #8 [internal] load build context
    #8 transferring context: 1.02MB 0.0s done
    #8 DONE 0.0s
    
    #6 [ 2/11] RUN useradd --create-home --shell /bin/bash calitp &&
      apt-get update &&
      apt-get install -qq --no-install-recommends gettext nginx &&
      mkdir -p /var/cache/nginx &&
      chown -R calitp /var/cache/nginx &&
      mkdir -p /var/lib/nginx &&
      chown -R calitp /var/lib/nginx &&
      mkdir -p /var/log/nginx &&
      chown -R calitp /var/log/nginx &&
      touch /var/run/nginx.pid &&
      chown -R calitp /var/run/nginx.pid &&
      mkdir -p /home/calitp/app/config &&
      mkdir -p /home/calitp/app/run &&
      mkdir -p /home/calitp/app/static &&
      chown -R calitp /home/calitp
    #6 CACHED
    
    #7 [ 3/11] WORKDIR /home/calitp/app
    #7 CACHED
    
    #9 [ 4/11] COPY requirements.txt requirements.txt
    #9 CACHED
    
    #10 [ 5/11] RUN pip install --no-cache-dir -r requirements.txt
    #10 CACHED
    
    ...

More information

Should get us closer to #133

Closes #134

EDIT: Workflow run after this PR was merged showing new build process

@thekaveman thekaveman added this to the October 2021 milestone Oct 9, 2021
@thekaveman thekaveman marked this pull request as ready for review October 14, 2021 16:45
@thekaveman thekaveman marked this pull request as draft October 14, 2021 16:50
@thekaveman thekaveman added this to In progress in Benefits 2021 via automation Oct 14, 2021
@thekaveman thekaveman removed this from In progress in Benefits 2021 Oct 14, 2021
@thekaveman thekaveman merged commit 85c56e2 into dev Oct 14, 2021
@thekaveman thekaveman deleted the actions/docker-build branch October 14, 2021 19:56
@thekaveman thekaveman mentioned this pull request Oct 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve Docker build in GitHub Actions
2 participants