Unable to run codspeed inside docker container

[finsberg]

Hi!

I am trying to use codspeed for the first time using GitHub action, but for some reason I cannot get it to work. One difference from the example online is that I need to run it inside a docker container. Here is the relevant PR: finsberg/fenicsx-pulse#39

Running the tests locally works fine.

Here is my GitHub actions .yml file

name: codspeed-benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    runs-on: ubuntu-22.04
    container: ghcr.io/fenics/dolfinx/dolfinx:nightly
    env:
      DEB_PYTHON_INSTALL_LAYOUT: deb_system

    steps:
      - uses: actions/checkout@v4

      - name: Install package needed by codspeed
        run: apt-get update && apt-get install -y curl

      - name: Install package
        run: python3 -m pip install -e .[benchmark]

      - name: Run benchmarks
        uses: CodSpeedHQ/action@v2
        with:
            
            token: ${{ secrets.CODSPEED_TOKEN }}
            run: python3 -m pytest tests/ --codspeed

Also, here are the logs from the run,

logs.txt

and it looks like the command

head_status=$(curl -I -fsSL -w "%{http_code}" -o /dev/null https://github.com/CodSpeedHQ/runner/releases/download/v$RUNNER_VERSION/codspeed-runner-installer.sh)

returns 404, which probably means that $RUNNER_VERSION is not set correctly. Any ideas on how to proceed?

[adriencaccia]

Hey!

I just looked at finsberg/fenicsx-pulse#39, and as you have figured out, the problem was due to the fact that there were some dependencies missing in the image you are using.

We quite heavily rely on the pre-installed software on the runner: https://github.com/actions/runner-images/blob/ubuntu22/20240630.1/images/ubuntu/Ubuntu2204-Readme.md

[finsberg]

Indeed, but it might be a good idea to add a section about this in the docs. In particular, that you need to run the container with the option --privileged, e.g (in my case)

jobs:
  benchmarks:
    runs-on: ubuntu-22.04
    container:
      image: ghcr.io/fenics/dolfinx/dolfinx:v0.8.0
      options: --privileged

This was not easy to figure out.

Also it would be nice if installing curl and lsb-release was also part of the action. Figuring out that I was missing curl was quite straight forward, but that I was missing lsb-release was not obvious to me.

[finsberg]

Looks like the problem is still not resolved. This is the output from the logs

2024-07-11T14:35:59.3399682Z ========== 1 passed, 73 deselected, 2 warnings in 2208.81s (0:36:48) ===========
2024-07-11T14:35:59.9642717Z ##[endgroup]
2024-07-11T14:35:59.9643331Z ##[group]Upload the results
2024-07-11T14:36:00.0987151Z ##[error]Error Failed to open repository at path: /__w/fenicsx-pulse/fenicsx-pulse/
2024-07-11T14:36:00.0997393Z ##[error]Process completed with exit code 1.
2024-07-11T14:36:00.1087509Z Post job cleanup.
2024-07-11T14:36:00.1093772Z ##[command]/usr/bin/docker exec  073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439 sh -c "cat /etc/*release | grep ^ID"
2024-07-11T14:36:00.3096645Z [command]/usr/bin/git version
2024-07-11T14:36:00.3138545Z git version 2.34.1
2024-07-11T14:36:00.3184394Z Temporarily overriding HOME='/__w/_temp/4fd6889d-1fd4-4675-9dfe-ea7acd493452' before making global git config changes
2024-07-11T14:36:00.3186146Z Adding repository directory to the temporary git global config as a safe directory
2024-07-11T14:36:00.3192415Z [command]/usr/bin/git config --global --add safe.directory /__w/fenicsx-pulse/fenicsx-pulse
2024-07-11T14:36:00.3228681Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2024-07-11T14:36:00.3261260Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2024-07-11T14:36:00.3527375Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-07-11T14:36:00.3552722Z http.https://github.com/.extraheader
2024-07-11T14:36:00.3566195Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2024-07-11T14:36:00.3598635Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2024-07-11T14:36:00.3992313Z Stop and remove container: 67352b11bbd5453ea936becb4c1eb710_ghcriofenicsdolfinxdolfinxv080_8a5e96
2024-07-11T14:36:00.3997778Z ##[command]/usr/bin/docker rm --force 073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439
2024-07-11T14:36:00.6497579Z 073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439
2024-07-11T14:36:00.6527041Z Remove container network: github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.6532007Z ##[command]/usr/bin/docker network rm github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.9109536Z github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.9287842Z Cleaning up orphan processes

Any ideas on what is missing here?

[adriencaccia]

Looks like the problem is still not resolved. This is the output from the logs

2024-07-11T14:35:59.9643331Z ##[group]Upload the results
2024-07-11T14:36:00.0987151Z ##[error]Error Failed to open repository at path: /__w/fenicsx-pulse/fenicsx-pulse/
2024-07-11T14:36:00.0997393Z ##[error]Process completed with exit code 1.

Any ideas on what is missing here?

I investigated and added a way to add more details about the error. I found this: https://github.com/adriencaccia/fenicsx-pulse/actions/runs/9905400860/job/27365652535#step:7:131
The user in the docker container does not own the repository, so when we want to open it in the CLI, it fails.

You can fix this by adding the following step in your workflow: adriencaccia/fenicsx-pulse@a595830 (#1)

I released https://github.com/CodSpeedHQ/action/releases/tag/v2.4.3 that allows displaying causes of errors when debug logging is enabled in actions

[finsberg]

Thanks a lot for the help 🙏

[Mark90]

I also ran into several issues when executing the codspeed action inside a container, 2 of which I could resolve with your help. A third issue I had was codspeed: command not found because for some reason the installer script was unable to add cargo's bin directory to the PATH.

It took me half a day to figure all of this out, so for anyone encountering this topic and wondering what is the complete set of fixes, this is what did it for me.

---

1. Run container with --privileged + (perhaps python specific) use non--slim image version

Before

    container: python:3.13-slim

After

    container:
      image: python:3.13
      options: --privileged

Thanks @finsberg for sharing this fix!

2. Ensure the container has curl

Perhaps obvious and not needed for all images, but the codspeed action needs curl to retrieve the installer script.

      - name: Install curl
        run: apt update && apt install curl -y

3. Add cargo bin directory to PATH

The installer script tried to add the bin directory (which contains codspeed) to the PATH but somehow this failed. So I did this manually:

      - name: Add .cargo/bin to PATH
        run: echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"

4. Allow codspeed to access my git repository files

A security feature from git/linux prevented codspeed from accessing the git repository. This happens at the end of the codspeed execution, when it tries to upload its results.

      - name: Fix git owner
        run: git config --global --add safe.directory "*"

Thanks @adriencaccia for sharing this fix!