cannot share action code for multiple runners

[oliverkurth]

Describe the bug

I am trying to create an RPM package for action runner for Linux The goal is to allow multiple runners to run in the same machine. This works when copying the complete package to separate directories. However, the directories bin and externals are huge, so to save disk space, these should be shared.

I tried to do this by installing all files from the release package into /usr/lib/github-runner. A script would copy all files into a directory in the github-runner users directory, except for bin and externals, which are made symlinks. See an example:

github-runner@photon-6cf5fcdc0a7f [ ~/runners/poi1 ]$ ls -la
total 44
drwxr-xr-x 2 github-runner users 4096 Mar 19 23:04 .
drwxr-xr-x 3 github-runner users 4096 Mar 19 22:59 ..
lrwxrwxrwx 1 github-runner users   26 Mar 19 23:03 bin -> /usr/lib/github-runner/bin
-rwxr-xr-x 1 github-runner users 2124 Mar 19 23:03 config.sh
-rw-r--r-- 1 github-runner users   17 Mar 19 22:59 .env
-rwxr-xr-x 1 github-runner users  646 Mar 19 23:03 env.sh
lrwxrwxrwx 1 github-runner users   32 Mar 19 23:03 externals -> /usr/lib/github-runner/externals
-rw-r--r-- 1 github-runner users   50 Mar 19 23:03 .path
-rw-r--r-- 1 github-runner users 1619 Mar 19 23:03 run-helper.cmd.template
-rwxr-xr-x 1 github-runner users 2323 Mar 19 23:03 run-helper.sh
-rwxr-xr-x 1 github-runner users 2323 Mar 19 23:03 run-helper.sh.template
-rwxr-xr-x 1 github-runner users 2195 Mar 19 23:03 run.sh
-rwxr-xr-x 1 github-runner users   65 Mar 19 23:03 safe_sleep.sh
github-runner@photon-6cf5fcdc0a7f [ ~/runners/poi1 ]$ 

However, this fails because the runner determines the real path, and tries to create the directory _diag in /usr/lib/github-runner which fails because the directory is not owned by github-runner. Even when the directory /usr/lib/github-runner/_diag is created, and owned by the github_runner user, the next error happens when the runner tries to create the file /usr/lib/github-runner/.credentials_rsaparams.

Even with permissions, the_diag directory and the file .credentials_rsaparams should be created under the runner directory, and not be shared among different runners.

To Reproduce
Steps to reproduce the behavior:

1. install actions runner files in /usr/lib/github-runner
2. crerte directory in a home directory
3. copy files to the directory, except bin and externals
4. create symlinks to /usr/lib/github-runner/bin and /usr/lib/github-runner/externals
5. run confi.sh

Expected behavior
Runner starts

Runner Version and Platform

2.322.0

Linux

What's not working?

Observe error:

Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5764]: ./config.sh: line 81:  5792 Aborted                 (core dumped) ./bin/Runner.Listener configure "$@"
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]: Unhandled exception. System.UnauthorizedAccessException: Access to the path '/usr/lib/github-runner/_diag' is denie>
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:  ---> System.IO.IOException: Permission denied
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    --- End of inner exception stack trace ---
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    at System.IO.FileSystem.CreateDirectory(String fullPath, UnixFileMode unixCreateMode)
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    at System.IO.Directory.CreateDirectory(String path)
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    at GitHub.Runner.Common.HostTraceListener..ctor(String logFileDirectory, String logFilePrefix, Int32 pageSizeLim>
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    at GitHub.Runner.Common.HostContext..ctor(String hostType, String logFile)
Mar 19 22:35:46 photon-6cf5fcdc0a7f runner.sh[5816]:    at GitHub.Runner.Listener.Program.Main(String[] args)

Job Log Output

see above

Runner and Worker's Diagnostic Logs

see above

[ChristopherHX]

As someone who found a practicable workaround for such usecase in a long forgotten past.

Additionally to symlinking files of the bin folder, I copy Runner. without file extension (or .exe windows), copy all dlls with Runner. prefix. To alter the .net root folder detection without code edits of this application.

See here my code I use for a long time now (used for azure pipelines agent as well) https://github.com/ChristopherHX/runner.server/blob/1dff97843ddce4730b8ea27940fef2d09e689e61/src/Runner.Client/Program.cs#L736-L772

Those files are not containing large data and are quick to copy, others are just like you tried symlinks.

BTW hardlinking can save space, since it only cloned the directory structure without being a symlink...

Somewhat offtopic, my fork contains code edits to allow changing this file location via an env variable as well, but uses the above logic for original binaries

[oliverkurth]

Thank you @ChristopherHX , I will try that as a workaround.

I thought about hard links, but that would require root access. Bind mounts probably work too, or bindfs. All of those require root access though (bindfs via fusermount with setuid bit set).