Document GitLab CI/CD

[oxr463]

Minimal .gitlab-ci.yml:

image: docker:latest

services:
  - docker:dind

build:
  script:
    - docker run --rm --volume "`pwd`:/data" --user `id -u`:`id -g` pandoc/latex README.md -o README.pdf
  artifacts:
    paths:
      - "*.pdf"

See: https://pandoc.org/installing.html#docker

[svenevs]

Hi @oxr463, I don't use / know gitlab. Is this the equivalent of "github actions" only for GitLab?

[oxr463]

Hi @oxr463, I don't use / know gitlab. Is this the equivalent of "github actions" only for GitLab?

Basically, except it's been around for much longer.

[tarleb]

@oxr463 would you like to make a PR to the docs so this won't get lost?

[cben]

Also explained in more details in https://medium.com/isovera/devops-for-presentations-reveal-js-markdown-pandoc-gitlab-ci-34d07d2c1011

[quizilkend]

You can find more information here:
https://gitlab.com/pandoc/pandoc-ci-example

[alerque]

As a long time GitLab CI user I have to suggest this doesn't look right at all. The setup in the issue a convoluted nested Docker invocation. I think the correct way to do this would be just setup the correct Pandoc host image in the first place. If it's setup properly I don't think you'll need any of the manual volume mounting or user mapping bits. I've done something similar to run CaSILE on GitLab CI and the result is quite a bit simpler. I'll look into how that would apply to these containers.

[tarleb]

@alerque I'll add you to the pandoc group on GitLab later. Seems like you are using the same alias as here?

[alerque]

Here is a way of doing this that I think is better.

This is similar to what's shown in PR №11 on pandoc-action-example where the entry point has been manipulated, but using GitLab CI instead of Actions:

default:
  image:
    name: "pandoc/latex:latest"
    entrypoint: [""]

pandoc:
  script:
    - pandoc README.md -o README.pdf
  artifacts:
    paths:
      - "*.pdf"

I'm sure there is another way to do with without the funky entry point hack but will have to test it in a sandbox first because I don't have a working project to copy from.

[alerque]

@tarleb Yes same alias for me, https://gitlab.com/alerque.

---

My experience is mostly with self-hosted GitLab instances an CI runners, so I'll have to check that their hosted version runs Docker images the same way. The above YAML sample works on my private self hosted instance as long as I use a Docker runner not a shell runner.

[alerque]

I see I'm late to the party ;-) The existing pandoc-ci-example repository on GitLab already has a working example roughly comparable to the one I suggested.

It looks like we should probably clarify each in the docs here and make sure the two also identify themselves and perhaps are crosslinked so people can discover the one that's best for them rather than re-inventing this wheel every time.

[bittner]

It looks like version 2.16.1 (commit edc2df5) broke the feature: 💣

$ docker run --rm --volume "`pwd`:/data" --user `id -u`:`id -g` --entrypoint /bin/sh pandoc/latex:2.16.1 -c "pandoc --version"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown.

The Docker image 2.16 still works:

$ docker run --rm --volume "`pwd`:/data" --user `id -u`:`id -g` --entrypoint /bin/sh pandoc/latex:2.16 -c "pandoc --version"
pandoc 2.16
Compiled with pandoc-types 1.22.1, texmath 0.12.3.2, skylighting 0.12.1,
citeproc 0.6, ipynb 0.1.0.2
User data directory: /.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

[tarleb]

I don't know what's happening there. I was not able to reproduce this locally.

[alerque]

@bittner That error you show is what happens when the arguments you are passing into Docker are not quoted properly. The example command you gave works with default bash (and zsh) options, but can easily break depending on shell options and will break if you try to nest that in a YAML config file to be passed to some shell runner (such as GitLab CI). I suggest you upgrade your usage habits a couple decades to newer style process substitution (i.e. use $(command) instead of `command`) and use variables instead of command substitution where possible (i.e. $PWD or ./ instead of $(pwd) if either of those happen to be viable). Additionally if you are just trying to run bare commands in the Docker container instead of setting a shell as the entry point and passing everything quoted in as -c 'command' args to the shell, just use your command as the entry point. Combining all these suggestions, your example becomes:

$ docker run --rm --volume "$PWD:/data" --user "$(id -u):$(id -g)" --entrypoint '' pandoc/latex:2.16.1 pandoc --version
pandoc 2.16.1
Compiled with pandoc-types 1.22.1, texmath 0.12.3.2, skylighting 0.12.1,
citeproc 0.6, ipynb 0.1.0.2
User data directory: /.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

This will be a much more robust starting point, especially for a command you're going to embed in another language such as YAML that will have it's own layer of quoting issues.

@tarleb I don't think there is anything actionable on this issue except updating the docs to reflect the actual GitLab example repository and the suggestions in above comments.

[tarleb]

Thanks @alerque, that makes sense. Closing due to the info above.

Side note: I still use backticks now and then, if only because they are convenient to type on some keyboard layouts. Of course, I usually regret it the moment I want nested program substitution.