From 2ba0385c95b258121cb1d16166d71b4c363c5110 Mon Sep 17 00:00:00 2001 From: Jason Walker Date: Mon, 27 Feb 2023 17:00:28 +0000 Subject: [PATCH 1/5] Added guide for GitLab CI --- _posts/2023-02-15-gitlab-ci.md | 150 +++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 _posts/2023-02-15-gitlab-ci.md diff --git a/_posts/2023-02-15-gitlab-ci.md b/_posts/2023-02-15-gitlab-ci.md new file mode 100644 index 00000000..38fe7d2a --- /dev/null +++ b/_posts/2023-02-15-gitlab-ci.md @@ -0,0 +1,150 @@ +--- +layout: post +title: "Working with GitLab CI" +author: "@raginjason" +authorUrl: https://github.com/raginjason +--- + +For simple use cases you can use your development container (devcontainer) for CI without much issue. Once you begin using more advanced devcontainer functionality such as [Features](/features), you will need devcontainer tooling in your CI pipeline. While GitHub CI has the [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), there is no such analog in GitLab CI. To achieve the goal of using your devcontainer in GitLab CI, the container must be pre-built. + +This document will guide you on how build a devcontainer with GitLab CI, push that devcontainer to the GitLab Container Registry, and finally reference that devcontainer in your main project for both local development and GitLab CI. + +For the purposes of this document, we will assume the main project is named `my-project` and lives under the `my-user` path. The example here uses a few [Features](/features), which is what forces the container to be pre-built. + +## The Development Container GitLab project + +Create a project in GitLab where the stand-alone devcontainer project will live. As the main project is assumed to be named `my-project`, let's assume the devcontainer project name will be `my-project-devcontainer` + +### Development Container .devcontainer/devcontainer.json + +The example here is a CDK project for Python makes use of both the [AWS CLI](https://github.com/devcontainers/features/tree/main/src/aws-cli) and [AWS CDK](http://github.com/devcontainers-contrib/features/tree/main/src/aws-cdk) layers. + +`.devcontainer/devcontainer.json`: + +```json +{ + "build": { + "context": "..", + "dockerfile": "Dockerfile" + }, + "features": { + "ghcr.io/devcontainers/features/aws-cli:1": {}, + "ghcr.io/devcontainers-contrib/features/aws-cdk:2": {} + }, + "customizations": { + "vscode": { + "settings": { + "python.formatting.provider": "black" + } + } + } +} +``` + +### Development Container Dockerfile + +As this is a Python project working with CDK, the `Dockerfile` will begin by using the latest Python dev container image and then install some basic packages via `pip`. + +`Dockerfile`: + +```Dockerfile +FROM mcr.microsoft.com/vscode/devcontainers/python:latest + +RUN pip3 --disable-pip-version-check --no-cache-dir install aws_cdk_lib constructs jsii pylint \ + && rm -rf /tmp/pip-tmp +``` + +### Development Container .gitlab-ci.yml + +Since there is no GitLab CI equivalent to [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), we will need to install the devcontainer CLI manually. The following will: + +1. Install the packages that the devcontainer CLI requires +2. Install the devcontainer CLI itself +3. Login to GitLab Container Repository +4. Build the dev container and push it to the GitLab Container Repository + +`.gitlab-ci.yml`: + +```yaml +image: docker:latest + +variables: + DOCKER_TLS_CERTDIR: "/certs" + +services: + - docker:dind + +before_script: + - apk add --update nodejs npm python3 make g++ + - npm install -g @devcontainers/cli + +build: + stage: build + script: + - docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY} + - devcontainer build --workspace-folder . --push true --image-name ${CI_REGISTRY_IMAGE}:latest +``` + +## The Main GitLab project + +### Main .devcontainer/devcontainer.json + +`.devcontainer/devcontainer.json`: + +```json +{ + "image": "registry.gitlab.com/my-user/my-project-devcontainer" +} +``` + +### Main .gitlab.ci.yml + +Assuming the devcontainer project name is based off the main project name, the `${CI_REGISTRY_NAME}` variable can be used. This configuration performs some basic sanity checks and linting once merge requests are submitted. + +`.gitlab-ci.json`: + +```yaml +image: ${CI_REGISTRY_IMAGE}-devcontainer:latest + +before_script: + - python --version + - cdk --version + +stages: + - Build + - Lint + +py_compile: + stage: Build + script: + - find . -type f -name "*.py" -print | xargs -n1 python3 -m py_compile + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + +cdk synth: + stage: Build + script: + - JSII_DEPRECATED=fail cdk --app "python3 app.py" synth + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + +Pylint: + stage: Lint + script: + - pylint * + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + +Black code format: + stage: Lint + script: + - black --check --diff . + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' +``` + +## Conclusion + +It's worth noting that the best practice would be to pin the versions of the various packages installed by `pip`, `apk`, `npm` and the like. Version pinning was omitted from this guide so that it can be executed as-is without issue. + +The above provides a starting point for a devcontainer that's used for both local development and in GitLab CI. It can easily be customized for other languages and tool chains. Take it and make it your own, happy coding! \ No newline at end of file From b0bf7cbc44135948a1b92786528510c0d555dfe5 Mon Sep 17 00:00:00 2001 From: Jason Walker Date: Mon, 27 Feb 2023 18:46:31 +0000 Subject: [PATCH 2/5] Grammar/spelling corrections for GitLab CI guide --- _posts/2023-02-15-gitlab-ci.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_posts/2023-02-15-gitlab-ci.md b/_posts/2023-02-15-gitlab-ci.md index 38fe7d2a..44f5d822 100644 --- a/_posts/2023-02-15-gitlab-ci.md +++ b/_posts/2023-02-15-gitlab-ci.md @@ -7,9 +7,9 @@ authorUrl: https://github.com/raginjason For simple use cases you can use your development container (devcontainer) for CI without much issue. Once you begin using more advanced devcontainer functionality such as [Features](/features), you will need devcontainer tooling in your CI pipeline. While GitHub CI has the [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), there is no such analog in GitLab CI. To achieve the goal of using your devcontainer in GitLab CI, the container must be pre-built. -This document will guide you on how build a devcontainer with GitLab CI, push that devcontainer to the GitLab Container Registry, and finally reference that devcontainer in your main project for both local development and GitLab CI. +This document will guide you on how to build a devcontainer with GitLab CI, push that devcontainer to the GitLab Container Registry, and finally reference that devcontainer in your main project for both local development and GitLab CI. -For the purposes of this document, we will assume the main project is named `my-project` and lives under the `my-user` path. The example here uses a few [Features](/features), which is what forces the container to be pre-built. +For the purpose of this document, we will assume the main project is named `my-project` and lives under the `my-user` path. The example here uses a few [Features](/features), which is what forces the container to be pre-built. ## The Development Container GitLab project From 7b71f1beabce98c267cf2433a56bd5a74c15a8d3 Mon Sep 17 00:00:00 2001 From: Jason Walker Date: Mon, 27 Feb 2023 18:48:53 +0000 Subject: [PATCH 3/5] Updated image path in GitLab CI guide example code --- _posts/2023-02-15-gitlab-ci.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2023-02-15-gitlab-ci.md b/_posts/2023-02-15-gitlab-ci.md index 44f5d822..f59ca14e 100644 --- a/_posts/2023-02-15-gitlab-ci.md +++ b/_posts/2023-02-15-gitlab-ci.md @@ -48,7 +48,7 @@ As this is a Python project working with CDK, the `Dockerfile` will begin by usi `Dockerfile`: ```Dockerfile -FROM mcr.microsoft.com/vscode/devcontainers/python:latest +FROM mcr.microsoft.com/devcontainers/python:latest RUN pip3 --disable-pip-version-check --no-cache-dir install aws_cdk_lib constructs jsii pylint \ && rm -rf /tmp/pip-tmp From fbb190a9db67eec81df0e695cff80cb5d73b15ca Mon Sep 17 00:00:00 2001 From: Jason Walker Date: Mon, 27 Feb 2023 18:55:23 +0000 Subject: [PATCH 4/5] Changed 'devcontainer' to 'dev container' in GitLab guide for consistency --- _posts/2023-02-15-gitlab-ci.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/_posts/2023-02-15-gitlab-ci.md b/_posts/2023-02-15-gitlab-ci.md index f59ca14e..5b0af243 100644 --- a/_posts/2023-02-15-gitlab-ci.md +++ b/_posts/2023-02-15-gitlab-ci.md @@ -5,17 +5,17 @@ author: "@raginjason" authorUrl: https://github.com/raginjason --- -For simple use cases you can use your development container (devcontainer) for CI without much issue. Once you begin using more advanced devcontainer functionality such as [Features](/features), you will need devcontainer tooling in your CI pipeline. While GitHub CI has the [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), there is no such analog in GitLab CI. To achieve the goal of using your devcontainer in GitLab CI, the container must be pre-built. +For simple use cases you can use your development container (dev container) for CI without much issue. Once you begin using more advanced dev container functionality such as [Features](/features), you will need dev container tooling in your CI pipeline. While GitHub CI has the [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), there is no such analog in GitLab CI. To achieve the goal of using your dev container in GitLab CI, the container must be pre-built. -This document will guide you on how to build a devcontainer with GitLab CI, push that devcontainer to the GitLab Container Registry, and finally reference that devcontainer in your main project for both local development and GitLab CI. +This document will guide you on how to build a dev container with GitLab CI, push that dev container to the GitLab Container Registry, and finally reference that dev container in your main project for both local development and GitLab CI. For the purpose of this document, we will assume the main project is named `my-project` and lives under the `my-user` path. The example here uses a few [Features](/features), which is what forces the container to be pre-built. -## The Development Container GitLab project +## The Development Container GitLab project -Create a project in GitLab where the stand-alone devcontainer project will live. As the main project is assumed to be named `my-project`, let's assume the devcontainer project name will be `my-project-devcontainer` +Create a project in GitLab where the stand-alone dev container project will live. As the main project is assumed to be named `my-project`, let's assume the dev container project name will be `my-project-dev-container` -### Development Container .devcontainer/devcontainer.json +### Development Container .devcontainer/devcontainer.json The example here is a CDK project for Python makes use of both the [AWS CLI](https://github.com/devcontainers/features/tree/main/src/aws-cli) and [AWS CDK](http://github.com/devcontainers-contrib/features/tree/main/src/aws-cdk) layers. @@ -41,7 +41,7 @@ The example here is a CDK project for Python makes use of both the [AWS CLI](htt } ``` -### Development Container Dockerfile +### Development Container Dockerfile As this is a Python project working with CDK, the `Dockerfile` will begin by using the latest Python dev container image and then install some basic packages via `pip`. @@ -54,12 +54,12 @@ RUN pip3 --disable-pip-version-check --no-cache-dir install aws_cdk_lib construc && rm -rf /tmp/pip-tmp ``` -### Development Container .gitlab-ci.yml +### Development Container .gitlab-ci.yml -Since there is no GitLab CI equivalent to [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), we will need to install the devcontainer CLI manually. The following will: +Since there is no GitLab CI equivalent to [devcontainers-ci GitHub Action](https://github.com/marketplace/actions/devcontainers-ci), we will need to install the devcontainers CLI manually. The following will: -1. Install the packages that the devcontainer CLI requires -2. Install the devcontainer CLI itself +1. Install the packages that the devcontainers CLI requires +2. Install the devcontainers CLI itself 3. Login to GitLab Container Repository 4. Build the dev container and push it to the GitLab Container Repository @@ -93,18 +93,18 @@ build: ```json { - "image": "registry.gitlab.com/my-user/my-project-devcontainer" + "image": "registry.gitlab.com/my-user/my-project-dev-container" } ``` ### Main .gitlab.ci.yml -Assuming the devcontainer project name is based off the main project name, the `${CI_REGISTRY_NAME}` variable can be used. This configuration performs some basic sanity checks and linting once merge requests are submitted. +Assuming the dev container project name is based off the main project name, the `${CI_REGISTRY_NAME}` variable can be used. This configuration performs some basic sanity checks and linting once merge requests are submitted. `.gitlab-ci.json`: ```yaml -image: ${CI_REGISTRY_IMAGE}-devcontainer:latest +image: ${CI_REGISTRY_IMAGE}-dev-container:latest before_script: - python --version @@ -147,4 +147,4 @@ Black code format: It's worth noting that the best practice would be to pin the versions of the various packages installed by `pip`, `apk`, `npm` and the like. Version pinning was omitted from this guide so that it can be executed as-is without issue. -The above provides a starting point for a devcontainer that's used for both local development and in GitLab CI. It can easily be customized for other languages and tool chains. Take it and make it your own, happy coding! \ No newline at end of file +The above provides a starting point for a dev container that's used for both local development and in GitLab CI. It can easily be customized for other languages and tool chains. Take it and make it your own, happy coding! \ No newline at end of file From 02b35db6cbdf18e52350a5df73f2d9746a234e60 Mon Sep 17 00:00:00 2001 From: Jason Walker Date: Mon, 27 Feb 2023 19:09:27 +0000 Subject: [PATCH 5/5] Added indication that CDK Feature is community maintained --- _posts/2023-02-15-gitlab-ci.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2023-02-15-gitlab-ci.md b/_posts/2023-02-15-gitlab-ci.md index 5b0af243..85104558 100644 --- a/_posts/2023-02-15-gitlab-ci.md +++ b/_posts/2023-02-15-gitlab-ci.md @@ -17,7 +17,7 @@ Create a project in GitLab where the stand-alone dev container project will live ### Development Container .devcontainer/devcontainer.json -The example here is a CDK project for Python makes use of both the [AWS CLI](https://github.com/devcontainers/features/tree/main/src/aws-cli) and [AWS CDK](http://github.com/devcontainers-contrib/features/tree/main/src/aws-cdk) layers. +The example here is a CDK project for Python makes use of both the [AWS CLI](https://github.com/devcontainers/features/tree/main/src/aws-cli) and the community-maintained [AWS CDK](http://github.com/devcontainers-contrib/features/tree/main/src/aws-cdk) Features. `.devcontainer/devcontainer.json`: