Publishing

Introduction

C2C CI utils is built to be able to publish a project with the following structure:

Have stabilization branches named by default <major>.<minor>. Have tag for the release named by default <major>.<minor>.<patch>.

With C2C CI utils, you can publish a python package and a Docker image from the same repository.

The default publishing is:

• Push on the <major>.<minor> branch will publish Docker images.
• Create the tag <major>.<minor>.<patch> will publish the Docker images, and the Python package.
• Push on a pull request will publish the Docker images with the pull request ID ad tag.
• Close a pull request will delete the related Docker images.
• Push on the master branch will publish the Docker images with the master tag (Publishing a python package is also possible).
• The version at the last line of the SECURITY.md of the master branch will also be published using the latest tag, this will respect the tags present in the configuration.
• In the SECURITY.md file of the master branch, we can also add a column Alternate Tag to publish the Docker images with another tag, this will respect the tags present in the configuration (only for Docker).

The Docker images are published on Docker Hub and GitHub Container Registry.

See also the publishing part of the Configuration.

CI integration

Add your project to the Geo GHCI GitHub application to manage the security:

• Trigger rebuild, then a dpkg version is updated.
• Update the dependency against the CVE.

Note

Requires to be GitHub organization owner ot application owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.

There are different ways to be able to publish an artifact:

• Directly with the GitHub Token, possible for anything that's on GitHub (Release, Docker images, ...)
• Using OpenID connect (OIDC), possible to publish on PyPI.
• Using other credentials available on gopass.

GitHub token

To be able to publish on GitHub Packages using the GitHub Token, you need the following permission in your workflow job:

permissions:
  package: write

OpenID connect (OIDC)

From version 1.7 you can publish on PyPI without any additional credentials, for that, you need the following permission in your workflow job:

permissions:
  id-token: write

Credentials from gopass

Add your project in the global secrets named CI_GPG_PRIVATE_KEYand GOPASS_CI_GITHUB_TOKEN.

Note

Requires to be GitHub organization owner, then ask for that on Slack to a GitHub owner like sbrunner or on the channel #c2c_support_github.

Step used to initialize gopass:

      # Get the secrets
      - uses: camptocamp/initialise-gopass-summon-action@v2
        with:
          ci-gpg-private-key: ${{secrets.CI_GPG_PRIVATE_KEY}}
          github-gopass-ci-token: ${{secrets.GOPASS_CI_GITHUB_TOKEN}}
          patterns: pypi docker
        if: env.HAS_SECRETS == 'HAS_SECRETS'

Pull request automation workflow

Get the workflow pull-request-automation.yaml to review automatically the CVE, dpkg and Renovate updates.

Docker image

You should create a ci/dpkg-versions.yaml file with {}:

echo {} > ci/dpkg-versions.yaml
git add ci/dpkg-versions.yaml

Add your Docker images in ci/config.yaml like:

publish:
  docker:
    images:
      - name: camptocamp/project

Publishing

c2cciutils should be added in ci/requirements.txt.

You can use this workflow to integrate it in the CI

env:
  HAS_SECRETS: ${{ secrets.HAS_SECRETS }}

jobs:
  main:
    ...

    permissions:
      ...

    steps:
      ...

      # Install c2cciutils
      - run: echo "${HOME}/.local/bin" >> ${GITHUB_PATH}
      - run: python3 -m pip install --user --requirement=ci/requirements.txt

      ...

      # Publish
      - name: Publish
        run: c2cciutils-publish
        if: env.HAS_SECRETS == 'HAS_SECRETS'

Note

The ci/dpkg-versions.yaml file is managed automatically, the only manual task to do is to remove old distributions wher we uses a new one.

Run locally

You can run the publishing locally in dry-run mode:

GITHUB_REF=... c2cciutils-publish --dry-run ...

Create version branch

When we create a new version branch, we should:

• Add a backport <version> label.
• Update the Renovate base branch.
• Update the SECURITY.md.
• Create the branch :-P

To create all of that, you can also run: c2cciutils-version --version=<version> [--force]

Note that didn't create a release

Create a release

Just create a tag:

git tag <release>
git pus origin <tag>