Skip to content

Commit

Permalink
Change CI to support tag for relases
Browse files Browse the repository at this point in the history
  • Loading branch information
@rascasoft
rascasoft committed Aug 4, 2023
1 parent a67d8cf commit b5fd3f9
Show file tree
Hide file tree
Showing 4 changed files with 219 additions and 148 deletions.
37 changes: 26 additions & 11 deletions .github/workflows/main.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@ name: Linter, test, build and push KPA container

env:
KPA_PROJECT: example
REGISTRY_NAME: ghcr.io
CONTAINER_NAME: mmul-it/kpa
CONTAINER_VERSION: latest
REGISTRY_GHCR: ghcr.io/mmul-it
REGISTRY_QUAY: quay.io/mmul
CONTAINER_NAME: kpa

on: [push]

jobs:
linter:
runs-on: ubuntu-latest
container:
image: ghcr.io/mmul-it/kpa-marp-pandoc
image: ghcr.io/mmul-it/kpa-marp-pandoc:v1.0.0
steps:
- name: Checkout repository
uses: actions/checkout@v3
Expand All @@ -30,7 +30,7 @@ jobs:
test:
runs-on: ubuntu-latest
container:
image: ghcr.io/mmul-it/kpa-marp-pandoc
image: ghcr.io/mmul-it/kpa-marp-pandoc:v1.0.0
needs: linter
steps:
- name: Checkout repository
Expand All @@ -41,7 +41,6 @@ jobs:
--roles-path ./playbooks/roles
- name: Generate markdown and pdf files
run: for KPA_DOC in $(ls projects/${KPA_PROJECT}/*.yml); do
echo "Processing ${TRAINING}";
ansible-playbook
-e @projects/${KPA_PROJECT}/common/settings.yml
-e @${KPA_DOC}
Expand All @@ -63,9 +62,25 @@ jobs:
needs: test
steps:
- uses: actions/checkout@v3
- name: Check whether github.ref_type is branch or tag
run: if [ "${{ github.ref_type }}" == 'branch' ];
then
echo "CONTAINER_VERSION=$(echo ${{ github.sha }} | cut -c 1-12)" >> ${GITHUB_ENV};
else
echo "CONTAINER_VERSION=$(echo ${{ github.ref_name }})" >> ${GITHUB_ENV};
fi
- name: Build the container image
run: docker build . --file Dockerfile --tag ${REGISTRY_NAME}/${CONTAINER_NAME}:${CONTAINER_VERSION}
- name: Login into the container registry
run: echo ${{ secrets.GHCR_TOKEN }} | docker login ghcr.io -u ${{ vars.GHCR_USER }} --password-stdin
- name: Push the image into the container registry
run: docker push ${REGISTRY_NAME}/${CONTAINER_NAME}:${CONTAINER_VERSION}
run: docker build .
--file Dockerfile
--tag ${REGISTRY_GHCR}/${CONTAINER_NAME}:${CONTAINER_VERSION}
--tag ${REGISTRY_GHCR}/${CONTAINER_NAME}:latest
--tag ${REGISTRY_QUAY}/${CONTAINER_NAME}:${CONTAINER_VERSION}
--tag ${REGISTRY_QUAY}/${CONTAINER_NAME}:latest
- name: Login into the GitHub Container Registry
run: echo "${{ secrets.GHCR_TOKEN }}" | docker login ${REGISTRY_GHCR} --username "${{ vars.GHCR_USER }}" --password-stdin
- name: Login into the Quay Container Registry
run: echo "${{ secrets.QUAY_ROBOT_TOKEN }}" | docker login ${REGISTRY_QUAY} --username "${{ vars.QUAY_ROBOT_NAME }}" --password-stdin
- name: Push the image into the GitHub Container Registry
run: docker push --all-tags ${REGISTRY_GHCR}/${CONTAINER_NAME}
- name: Push the image into the Quay Container Registry
run: docker push --all-tags ${REGISTRY_QUAY}/${CONTAINER_NAME}
135 changes: 63 additions & 72 deletions CI.md
View file
Original file line number Diff line number Diff line change
@@ -1,91 +1,76 @@
# Using KPA in CI

KPA can be used in a CI context to automate the production of slide sets after each push to your repository.
KPA can be used in a CI context to automate the production of slide sets after
each push to your repository.

While producing new *Knowleldge Pods* or modifying existing ones you will have an automated way to get your trainings or presentations ready and versioned each time you modify something.
While producing new *Knowleldge Pods* or modifying existing ones you will have
an automated way to get your trainings or presentations ready and versioned each
time you modify something.

## How to make it work

To implement CI in KPA a container named [kpa is available at quay.io](https://quay.io/repository/mmul/), complete of this repository, the `kpa_generator` Ansible role, and the Marp and Pandoc tools. It is everything you need to automate the generation.
To implement CI in KPA a container named **kpa** is available at [ghcr.io/mmul-it/kpa](https://ghcr.io/mmul-it/kpa),
containing the `kpa_generator` Ansible role, the Marp, and Pandoc tools.

Your KPA project will live in a repository and should have the structure described in the [Create a KPA project](https://github.com/mmul-it/kpa/#create-a-kpa-project) chapter inside the main documentation file of this repository.
It is everything you need to automate the generation.

Your KPA project will live in a repository and should have the structure
described in the [Create a KPA project](https://github.com/mmul-it/kpa/#create-a-kpa-project)
chapter inside the main documentation file of this repository.

## Implement CI with GitHub Actions

Once you're done creating the structure, if you want to [automate your workflow with GitHub Actions](https://github.com/features/actions), you will need to create a file under the `.github/workflows` path, named for example `main.yml`, with these contents:
Once you're done creating the structure, if you want to automate your workflow
with [GitHub Actions](https://github.com/features/actions), you will need to
create a file under the `.github/workflows` path, named for example `main.yml`,
with these contents:

```yaml
name: Automate PDF file creation
name: Check and generate markdown and PDF files for mmul project

env:
KPA_PROJECT: myproject
KPA_PROJECT: kpa-mmul

on: [push]

defaults:
run:
working-directory: /kpa

jobs:
lint:
runs-on: ubuntu-latest
container:
image: quay.io/mmul/kpa
image: ghcr.io/mmul-it/kpa
options: --user root
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Link project dir in KPA projects folder
run: ln -vs ${GITHUB_WORKSPACE} /kpa/projects/${KPA_PROJECT}
- name: Create symlink to workspace
run: ln -sf /__w/kpa-mmul/kpa-mmul /kpa/projects/${KPA_PROJECT}
- name: Check project yamls
run: yamllint -s projects/${KPA_PROJECT}/*.yml
- name: Check markdown files for example project
run: mdl projects/${KPA_PROJECT}/contents/*.md
run: yamllint -s /kpa/projects/${KPA_PROJECT}/*.yml
- name: Check markdown files for example project
run: mdl /kpa/projects/${KPA_PROJECT}/contents/*.md

markdown:
generate:
runs-on: ubuntu-latest
needs: lint
container:
image: quay.io/mmul/kpa
image: ghcr.io/mmul-it/kpa
options: --user root
needs: lint
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Link project dir in KPA projects folder
run: ln -vs ${GITHUB_WORKSPACE} /kpa/projects/${KPA_PROJECT}
- name: Generate markdown files
run: for TRAINING in $(ls projects/${KPA_PROJECT}/*.yml); do
echo "Processing ${TRAINING}";
ansible-playbook
-e @${TRAINING}
playbooks/kpa_generator.yml;
- name: Create symlink to workspace
run: ln -sf /__w/kpa-mmul/kpa-mmul /kpa/projects/${KPA_PROJECT}
- name: Generate files
run: for DOC in $(ls /kpa/projects/${KPA_PROJECT}/*.yml); do
echo "Processing $(basename ${DOC})";
/kpa/kpa.sh -p ${KPA_PROJECT} -y $(basename ${DOC});
done
- run: ls output
- name: Upload markdown files
uses: actions/upload-artifact@v3
with:
name: markdowns
path: /kpa/output/**.md

marp-pdf:
runs-on: ubuntu-latest
needs: markdown
container:
image: quay.io/mmul/kpa
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Link project dir in KPA projects folder
run: ln -vs ${GITHUB_WORKSPACE} /kpa/projects/${KPA_PROJECT}
- name: Download markdown files
uses: actions/download-artifact@v3
with:
name: markdowns
path: /kpa/output/
- name: Generate Marp pdf files
run: for TRAINING in $(ls output/*.md); do
echo "Processing ${TRAINING}";
marp --theme projects/${KPA_PROJECT}/common/theme.css --html --pdf --allow-local-files ${TRAINING};
done
- name: Upload Marp pdf files
- name: Upload pdf files
uses: actions/upload-artifact@v3
with:
name: pdfs
Expand All @@ -94,29 +79,31 @@ jobs:

Each step is self explaining, but to give some highlights:

- There's an overall environment variable definition, `KPA_PROJECT`, that will be used in all the jobs.

- There's an overall environment variable definition, `KPA_PROJECT`, that will
be used in all the jobs.
- The CI will be launched for each push operation (`on: [push]`).

- The working directory of all the jobs will be `/kpa`, available in the `quay.io/mmul/kpa` container.

- There are three jobs that will check:

- The yaml linter consistency of the definition (`lint`).

- The markdown files generation (`markdown`).
- The pdf files generation (`marp-pdf`).
Each step is dependent from the previous one and starts with `Link project dir in KPA projects folder`, a trick to put your project available in the `/kpa/projects` folder.

**Note**: that this repository itself uses GitHub actions to automatically lint the Ansible code and the yaml and then produce slides and agenda for the default `example` project, check [.github/workflows/main.yml](.github/workflows/main.yml).

**Note**: the KPA project was presented in several public occasions using a presentation that is available at [https://github.com/mmul-it/kpa-mmul](https://github.com/mmul-it/kpa-mmul). This KPA project includes [a GitHub Workflow pipeline](https://github.com/mmul-it/kpa-mmul/blob/main/.github/workflows/main.yml) that generates both the slides and the agenda for each commit.
- There are two main jobs that will check:
- The yaml and markdown linter consistency (`lint`).
- The Ansible playbook execution, that will generate the files (`generate`)
and push them as artifacts into GitHub.
Each step is dependent from the previous one and starts with `Link project dir
in KPA projects folder`, a trick to put your project available in the
`/kpa/projects` folder.

**Note**: this same repository uses GitHub actions to automatically lint the
Ansible code and the yaml and then produce slides and agenda for the default
`example` project, check [.github/workflows/main.yml](.github/workflows/main.yml).

**Note**: the KPA project was presented in several public occasions using a
presentation that is available at [https://github.com/mmul-it/kpa-mmul](https://github.com/mmul-it/kpa-mmul).
This KPA project includes [the GitHub Workflow pipeline](https://github.com/mmul-it/kpa-mmul/blob/main/.github/workflows/main.yml)
mentioned in the above code portion.

## Implement CI with GitLab CI

If your repository lives inside GitLab then this is a practical example about how to use the KPA container to process a project named `myproject` using GitLab CI.
If your repository lives inside GitLab then this is a practical example about
how to use the KPA container to process a project named `myproject` using GitLab
CI.

A `.gitlab-ci.yml` yaml file should contain something like this:

Expand Down Expand Up @@ -182,6 +169,10 @@ marp-pdf:
done
```

It uses the same trick of the symbolic link as the GitHub's one and it does essentially the same three main steps, or stage as GitLab calls them.
It uses the same trick of the symbolic link as the GitHub's one and it does
essentially the same three main steps, or stage as GitLab calls them.

This file will be placed into your project's GitLab repository and, for every push, will look for every training or content yaml file definition and process it into a pdf file under the `output` directory, making those files available as GitLab job's artifacts.
This file will be placed into your project's GitLab repository and, for every
push, will look for every training or content yaml file definition and process
it into a pdf file under the `output` directory, making those files available as
GitLab job's artifacts.
4 changes: 2 additions & 2 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Dockerfile for KPA project to be used in CI

# Start from ansible-core
FROM quay.io/mmul/kpa-marp-pandoc
# Start from kpa-marp-pandoc
FROM ghcr.io/mmul-it/kpa-marp-pandoc:v1.0.0

# Create workdir path
RUN mkdir /kpa
Expand Down
Loading

0 comments on commit b5fd3f9

Please sign in to comment.