Provide description of requirements for running Unity CI actions on a self-hosted runner

[Vivraan]

Is your feature request related to a problem? Please describe.
Currently afaik Windows and other OSs are unsupported so I can't use a self-hosted runner without knowing about the capabilities expected by this action (or I need to install Ubuntu beside Windows on my system which may be undesirable unless the GitHub self-hosted runner application works through WSL, which is harder to set up than on a Linux distro which supports systemd).

Describe the solution you'd like
A description page which provides clear information on what configuration is expected from a machine which shall run this action.

Describe alternatives you've considered
The only other alternative is to go into the repo and dive into the code to find what capabilities are expected.

Additional context
When I first tried running the Activation action using windows-latest I was greeted with an error which read something like win32 is not supported.

[webbertakken]

Thank you @Vivraan for this valuable feedback.

I agree we should add a section about self-hosted runners, and expected capabilities, which simply put is ubuntu, and the configuration the same as default runners. What other information would you expect to find on this page?

I think the biggest problem with Windows specifically would be doubling the code for things like file paths and other OS specific things. In practice it wouldn't be too hard to support Windows at all, as the action is mostly a bootstrapper that mounts a container.

Looking forward to hear your thoughts on this.

[Vivraan]

I agree we should add a section about self-hosted runners, and expected capabilities, which simply put is ubuntu, and the configuration the same as default runners. What other information would you expect to find on this page?

Perhaps a clarifying link to the config of default runners would suffice.

I think the biggest problem with Windows specifically would be doubling the code for things like file paths and other OS specific things. In practice it wouldn't be too hard to support Windows at all, as the action is mostly a bootstrapper that mounts a container.

What would be required for the action to run on Ubuntu or a supported distro through WSL (apart from the custom setup without systemd)? Can it be done in a sane way?
We should also consider the case of macOS and other Linux distros.

Where can I change the OS specific stuff? I'll need to take a look at the docs, in case I can make time for contributing! - but no guarantees. Looks to me like we'll just need to add a .bat or .ps1 script for Windows.

[webbertakken]

Perhaps a clarifying link to the config of default runners would suffice.

Sounds good, lets do that.

What would be required for the action to run on Ubuntu or a supported distro through WSL (apart from the custom setup without systemd)? Can it be done in a sane way?

As far as I understand if it uses the shell all commands should work. The only thing that needs to happen is for the linux check to be removed I think.

For MacOS I expect it to work out of the box, unless paths differ, which I don't think they do for Unity, but i'm not sure.

Since actions require either a Docker or a Nodejs entrypoint, I don't see why we would need any shell scripts like a ps1. Perhaps I'm missing something?

Since the action is based on Nodejs I don't think there's a need for any powershell. Most should work out of the box. Perhaps just have a look at the source of the action, specifically entrypoint.sh and the src folder. The code should be self documenting.

[Vivraan]

For MacOS I expect it to work out of the box, unless paths differ, which I don't think they do for Unity, but i'm not sure.

If I understood correctly, you're asking if the Application.persistentDataPath differs for macOS, which indeed does: https://docs.unity3d.com/2018.4/Documentation/ScriptReference/Application-persistentDataPath.html

What I said there is irrelevant.

Since actions require either a Docker or a Nodejs entrypoint, I don't see why we would need any shell scripts like a ps1. Perhaps I'm missing something?

Since the action is based on Nodejs I don't think there's a need for any powershell. Most should work out of the box. Perhaps just have a look at the source of the action, specifically entrypoint.sh and the src folder. The code should be self documenting.

If we don't work with WSL, we can't use bash scripts, which is why I brought up batch and powershell scripting. Otherwise, we should have no problems with just bash scripts.

However, I should be clarifying that we need to do this for all the actions, so I'll change the title of the issue to match.

Update

From what I found in the entrypoint.sh files in Unity request activation file and Unity activate the path separators aren't important. Let's go through the other actions and find usages of OS specific paths.

Btw an assistance question: how would I publish the output of the Build action under the Releases tab?

[Vivraan]

Would you like to answer this part of the issue in Unity Actions instead?

[Vivraan]

Perhaps a clarifying link to the config of default runners would suffice.

Found this from the logs form GitHub Actions: https://github.com/actions/virtual-environments/blob/ubuntu18/20200920.1/images/linux/Ubuntu1804-README.md

[HaSa1002]

I am gonna leave our custom setup here as a reference for other people trying to get theirs up to speed. It works out of the box with Linux (we use Debian). The Windows setup was a bit of a stretch though. Our Windows Server is managed by Proxmox and enabling Hyper-V resulted in the server refusing to start. Because of that, we build the docker containers ourselves as part of the workflow run on Windows as the offical ones require a different Windows version.

Workflow.yml

# Copied and adapted from https://game.ci/docs/github/getting-started#workflow-examples
# We have three scripts, one for PRs, one for main builds and one for stable builds to not run macos everytime.
name: Unity Build

on:
  push:
    branches:
      - main
      - release/*
  pull_request:

concurrency:
  group: ci-${{github.ref}}-test
  cancel-in-progress: true

env:
  ITCH_USERNAME: <your username>
  ITCH_GAME_ID: <your game>
  BUTLER_API_KEY: ${{ secrets.BUTLER_API_KEY }}

jobs:
  buildAndTestForLinuxBasedPlatforms:
    name: Build for ${{ matrix.targetPlatform }}
    timeout-minutes: 60 # Windows builds tend to hang on Unity exit
    strategy:
      fail-fast: false # We suffer heavily from license acquisition fails. This lets working builds alive.
      matrix:
        include:
          - targetPlatform: StandaloneLinux64
            os: [self-hosted, linux]
            channel: linux-x64-stable
          - targetPlatform: StandaloneWindows64
            os: [self-hosted, windows]
            channel: win-x64-stable
          - targetPlatform: StandaloneOSX
            os: macos-latest
            channel: mac-stable
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout Branch
        uses: actions/checkout@v3

      - name: Test ${{ matrix.os }} (${{ runner.os }}) for ${{ matrix.targetPlatform }} in ${{ matrix.channel }} triggered by ${{ github.event_name }}
        run: git status

      # Git LFS
      - name: Create LFS file list
        run: git lfs ls-files -l | cut -d' ' -f 1 | sort > .lfs-assets-id

      - name: Restore LFS cache
        uses: actions/cache@v3
        id: lfs-cache
        with:
          path: .git/lfs
          key: ${{ runner.os }}-lfs-${{ hashFiles('.lfs-assets-id') }}
          restore-keys: |
            ${{ runner.os }}-lfs-

      - name: Git LFS Pull
        run: |
          git lfs pull
          git add .
          git reset --hard

      - name: Restore Library Cache
        uses: actions/cache/restore@v3
        with:
          path: Library
          enableCrossOsArchive: true
          key:
            Library-${{ matrix.targetPlatform }}-${{hashFiles('Library') }}
          restore-keys: |
            Library-${{ matrix.targetPlatform }}-
            Library-

      - name: Windows - Get Project Version
        id: unityVersion
        if: runner.os == 'Windows'
        run: |
          $version = ((Get-Content "ProjectSettings/ProjectVersion.txt")[0] -split ': ')[1]
          $(echo v=$version) >> $env:GITHUB_OUTPUT

      - name: Windows - Build Docker Base Image
        if: runner.os == 'Windows'
        run: docker build https://github.com/game-ci/docker.git#main:images/windows/base --build-arg windowsImage="mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2022" -t unityci/base:22
      
      - name: Windows - Build Unity Hub Image
        if: runner.os == 'Windows'
        run: docker build https://github.com/game-ci/docker.git#main:images/windows/hub --build-arg baseImage="unityci/base:22" -t unityci/hub:22

      - name: Windows - Build Unity Editor Image
        if: runner.os == 'Windows'
        run: docker build https://github.com/game-ci/docker.git#main:images/windows/editor --build-arg baseImage="unityci/base:22" --build-arg hubImage="unityci/hub:22" --build-arg version="${{ steps.unityVersion.outputs.v }}" --build-arg module=windows-il2cpp -t unityci/editor:22-windows-${{ steps.unityVersion.outputs.v }}-windows-il2cpp

      - name: Windows - Build Project
        uses: game-ci/unity-builder@v2
        if: contains(matrix.os, 'windows')
        id: build-win
        env:
          UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
          UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
          UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
        with:
          targetPlatform: ${{ matrix.targetPlatform }}
          chownFilesTo: "1000:1000"
          customImage: unityci/editor:22-windows-${{ steps.unityVersion.outputs.v }}-windows-il2cpp

      - name: Build Project
        uses: game-ci/unity-builder@v2
        if: contains(matrix.os, 'linux') || contains(matrix.os, 'macos-latest')
        id: build
        env:
          UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
          UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
          UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
        with:
          targetPlatform: ${{ matrix.targetPlatform }}
          chownFilesTo: "1000:1000"

      - name: Run Tests
        uses: game-ci/unity-test-runner@v2
        if: contains(matrix.os, 'linux')
        id: testRunner
        env:
          UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
          UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
          UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
        with:
          githubToken: ${{ secrets.GITHUB_TOKEN }}
          chownFilesTo: "1000:1000"

      - name: Save Test Output
        uses: actions/upload-artifact@v3
        if: contains(matrix.os, 'linux') && always()
        with:
          name: Test results (all modes)
          path: ${{ steps.testRunner.outputs.artifactsPath }}

      - name: Clear Test Output
        if: contains(matrix.os, 'linux') && always()
        run: sudo rm -rf ${{ steps.testRunner.outputs.artifactsPath }}
        continue-on-error: true

      - name: Store Library Cache
        uses: actions/cache/save@v3
        with:
          enableCrossOsArchive: true
          path: Library
          key: Library-${{ matrix.targetPlatform }}-${{hashFiles('Library') }}
      
      # Installing butler late, because build fails as dirty otherwise
      # Installing butler individually on each OS as the actions use containers that only work on Linux
      # It's faster and doing this way, though ugly, still leads to common callability
      - name: Install butler on Linux
        if: contains(matrix.os, 'linux') && (github.event_name == 'push' || github.event_name == 'release')
        run: |
          curl -L -o butler.zip https://broth.itch.ovh/butler/linux-amd64/LATEST/archive/default
          unzip butler.zip
          chmod +x butler
          ./butler -V
      
      - name: Install butler on Windows
        if: contains(matrix.os, 'windows') && github.event_name == 'push'
        run: |
          $progresspreference = 'silentlyContinue'
          Invoke-WebRequest https://broth.itch.ovh/butler/windows-amd64/LATEST/archive/default -OutFile butler.zip
          Expand-Archive butler.zip .
          $progressPreference = 'Continue'
          ./butler -V
      
      - name: Install butler on MacOS
        if: matrix.os == 'macos-latest' && github.event_name == 'push'
        run: |
          curl -L -o butler.zip https://broth.itch.ovh/butler/darwin-amd64/LATEST/archive/default
          unzip butler.zip
          chmod +x butler
          ./butler -V

      - name: Publish to Itch.io
        if: github.event_name == 'push'
        run: "./butler push ./build/${{ matrix.targetPlatform }} ${{env.ITCH_USERNAME}}/${{ env.ITCH_GAME_ID }}:${{ matrix.channel }} --userversion=${{ contains(matrix.os, 'windows') && steps.build-win.outputs.buildVersion || steps.build.outputs.buildVersion }}"

[m1rl0k]

hello running into issues trying to do this on a windows self hosted runner; any updates on this??

[m1rl0k]

soecifically for unity 23.... the docker images are no longer valid in the above workflow as it fails finding them

[HaSa1002]

What do you do specifically? I had to select the appropriate windows docker base images depending on the version