CI Recipes

CI Recipes

Atlas needs a real Vintage Story server install to run its E2E suite: the game files are not redistributable, so every CI job downloads them from the official CDN and points VINTAGE_STORY at the extracted VintagestoryAPI.dll folder.

The recipe below is trimmed from Atlas's own .github/workflows/ci.yml, adapted for a consumer project (a mod's own test suite) rather than Atlas itself.

Full recipe

name: ci

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
  VS_VERSION: "1.22.2"

jobs:
  build-and-pure-tests:
    name: Build & pure tests
    runs-on: ubuntu-latest
    timeout-minutes: 10

    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET 10
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 10.0.x

      - name: Cache Vintage Story server
        id: cache-vs
        uses: actions/cache@v4
        with:
          path: ~/vs
          key: vs-server-linux-x64-${{ env.VS_VERSION }}

      - name: Download Vintage Story server
        if: steps.cache-vs.outputs.cache-hit != 'true'
        run: |
          set -e
          curl -sSL --proto '=https' --proto-redir '=https' -o vs_server.tar.gz \
            "https://cdn.vintagestory.at/gamefiles/stable/vs_server_linux-x64_${VS_VERSION}.tar.gz"
          mkdir -p "$HOME/vs"
          tar -xzf vs_server.tar.gz -C "$HOME/vs"

      - name: Export VINTAGE_STORY
        run: |
          set -e
          DLL_DIR="$(dirname "$(find "$HOME/vs" -name VintagestoryAPI.dll | head -1)")"
          echo "VINTAGE_STORY=$DLL_DIR" >> "$GITHUB_ENV"

      - name: Build
        run: dotnet build YourSolution.sln -c Release

      # Tests that do not need a live embedded server: fast feedback, no VS download needed
      # if you split this into its own job without the VS setup steps.
      - name: Pure tests
        run: |
          dotnet test tests/YourMod.Pure.Tests \
            -c Release \
            --filter "Category!=E2E" \
            --logger "trx;LogFileName=pure-tests.trx" \
            --results-directory TestResults

      - name: Upload pure test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: pure-tests-trx
          path: TestResults/pure-tests.trx

  e2e:
    name: E2E scenarios (VS ${{ matrix.vs-version }})
    runs-on: ubuntu-latest
    timeout-minutes: 15
    needs: build-and-pure-tests
    strategy:
      fail-fast: false
      matrix:
        vs-version: ["1.22.0", "1.22.1", "1.22.2"]

    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET 10
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 10.0.x

      - name: Cache Vintage Story server
        id: cache-vs
        uses: actions/cache@v4
        with:
          path: ~/vs
          key: vs-server-linux-x64-${{ matrix.vs-version }}

      - name: Download Vintage Story server
        if: steps.cache-vs.outputs.cache-hit != 'true'
        run: |
          set -e
          curl -sSL --proto '=https' --proto-redir '=https' -o vs_server.tar.gz \
            "https://cdn.vintagestory.at/gamefiles/stable/vs_server_linux-x64_${{ matrix.vs-version }}.tar.gz"
          mkdir -p "$HOME/vs"
          tar -xzf vs_server.tar.gz -C "$HOME/vs"

      - name: Export VINTAGE_STORY
        run: |
          set -e
          DLL_DIR="$(dirname "$(find "$HOME/vs" -name VintagestoryAPI.dll | head -1)")"
          echo "VINTAGE_STORY=$DLL_DIR" >> "$GITHUB_ENV"

      - name: E2E tests
        run: |
          dotnet test tests/YourMod.E2E.Tests \
            -c Release \
            --filter "Category=E2E" \
            --logger "trx;LogFileName=e2e-tests.trx" \
            --results-directory TestResults

      - name: Upload E2E test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: e2e-tests-trx-${{ matrix.vs-version }}
          path: TestResults/*.trx

Notes

• Pure vs E2E filters: split your test suite into a fast "pure" category (no live server, runs on every push) and an "E2E" category (--filter "Category=E2E", needs VINTAGE_STORY and a real embedded server boot). This mirrors Atlas's own split between Atlas.Pure.Tests and Atlas.Engine.Tests.

• Version matrix pattern: run the E2E job across a small matrix of recent patch versions (fail-fast: false so one leg failing does not hide the others), separate from a full compatibility sweep across major/minor versions (see Compatibility).

• TRX output: --logger "trx;LogFileName=..." plus --results-directory gives every job a standard dotnet test TRX file, uploaded as a build artifact for later inspection; nothing custom is needed for reporting.

• Parallel execution: the E2E step can swap dotnet test for the atlas CLI's multi-process orchestrator:

  dotnet build tests/YourMod.E2E.Tests -c Release
  atlas run tests/YourMod.E2E.Tests/bin/Release/net10.0/YourMod.E2E.Tests.dll \
    --parallel --trx TestResults/e2e-tests.trx

  One worker subprocess per scenario class, and --trx writes one aggregated VSTest-style TRX report, so the artifact upload step above keeps working unchanged. Mind the tradeoff on small runners: each worker boots its own embedded server and wants roughly two cores, which is why the default worker count is half the cores. On a 2-core hosted runner that default is 1 worker, so --parallel buys nothing over dotnet test there; it pays off on larger self-hosted or paid runners with several scenario classes. See CLI.

• The shutdown flake: Vintage Story 1.22.2 occasionally throws a NullReferenceException from ServerSystemMonitor.Dispose() during embedded server teardown. Atlas catches and swallows this specific failure; it does not fail your test run and is not something CI needs to work around. See Troubleshooting and issue #8.

• Job timeouts: set an explicit timeout-minutes on every job that boots a real server. A stuck embedded server boot (bad VINTAGE_STORY, corrupted download) should fail the job in minutes, not consume the whole Actions timeout budget.

• Caching: the Vintage Story server archive rarely changes for a pinned version, so actions/cache keyed on the version string avoids re-downloading it on every run.