Ability to exit runner svc after completing workflow

[tabossert]

Describe the enhancement
I would like to be able to set a config option that would tell the runner to exit after it runs a workflow. My use case is having a container with the runner inside, when a workflow completes the runner exits causing container to restart, which would provide a fresh environment for each workflow.

[bryanmacfarlane]

We're calling this ephemeral runners and it's something we're looking at now. We're also looking at the service being cooperative in this scenario (service doesn't assign more work, runner exits, service auto cleans up registration).

If it's OK with you, I'll rebrand this ephemeral ...

[tabossert]

Sounds great, Thanks!

Happy to be an early tester when possible.

[adiroiban]

Looking forward to this feature.
I guess that the runner can send a "quit/shutdown" message to GitHub Actions server to have a clean shutdown.

Having the option to call an exterrnal command or HTTP endpoint outside of the workflow configuration file and for all workflows would be perfect.
In this way I can hook the VM shutdown or make an external webhook to a provisioning service which could shut the VM and restart it from a snapshot/savepoint.

@tabossert if you are on a private repo without PR from untrusted forks you can setup a workflow that will make a call to a local suicide command :)

My problem is that for public open source repos is easy to replace the workflow yaml file in the fork.

[timharris777]

Doesn't this already exist with the run.sh --once option?

[tabossert]

@timharris777 I believe I tried that and while it only processes a single workflow, the service itself does not exit. In a Docker container this means PID 1 doesn't get SIGTERM so container remains up.

@adiroiban I tried that, but it causes workflow to never finish as the "Complete Job" isn't able to run, It might be possible to background the shutdown command with a sleep.

[timharris777]

@tabossert , that's interesting as I am running this in AWS ECS Fargate with the --once option and it seems to work just fine.

Here is our .sh script that is used as the entrypoint on the docker image:

#!/usr/bin/env bash

# Create the runner and start the configuration experience
./config.sh --version
echo "starting...$NAME"

# get token
token=$(curl -s -XPOST \
    -H "authorization: token ${PAT}" \
    "https://api.github.com/repos/${ORG}/${REPO}/actions/runners/registration-token" |\
    jq -r .token)
if [ -z "$LABELS" ]
then
      echo "No labels specified for runner. Continuing on with [self-hosted]..."
      LABEL_COMMAND=""
else
      echo "Labels specified for runner. Continuing on with [self-hosted,${LABELS}]..."
      LABEL_COMMAND="--labels ${LABELS}"
fi


# if REGISTER_ONLY is set, register and exit
# if REMOVE is set, remove runner from reop exit
# else we are running
if [ "${REGISTER_ONLY}" == "TRUE" ]; then
    echo "Registering runner with repository..."
    ./config.sh --url "https://github.com/${ORG}/${REPO}" --token "${token}" --name "${NAME}" --work _work ${LABEL_COMMAND}
    echo "registration only done"
elif [ "${REMOVE}" == "TRUE" ]; then
    echo "Removing registered runner..."
    ls -lrt .
    ./config.sh remove --token "${token}"
    echo "remove action runner done"
else
    echo "Configuring runner with repository..."
    ./config.sh --url "https://github.com/${ORG}/${REPO}" --token "${token}" --name "${NAME}" --work _work ${LABEL_COMMAND}

    # run
    if [ "${EPHEMERAL}" == "TRUE" ]
    then
        echo "Running ephemeral (one-time) runner..."
        ./run.sh --once
    else
        echo "Running always on runner..."
        ./run.sh
    fi
    

    # remove
    echo "Removing one time runner..."
    ./config.sh remove --token "${token}"
    echo "Complete!"

[tabossert]

Thanks @timharris777 I will try that out!

[adiroiban]

When was the --once option introduced?

I don't see it available

$ ./bin.2.263.0/Runner.Listener --once

Commands:
 ./config.sh         Configures the runner
 ./config.sh remove  Unconfigures the runner
 ./run.sh            Runs the runner interactively. Does not require any options.

Options:
 --help     Prints the help for each command
 --version  Prints the runner version
 --commit   Prints the runner commit

Config Options:
 --unattended     Disable interactive prompts for missing arguments. Defaults will be used for missing options
 --url string     Repository to add the runner to. Required if unattended
 --token string   Registration token. Required if unattended
 --name string    Name of the runner to configure (default ins-13)
 --labels string  Extra labels in addition to the default: 'self-hosted,Linux,X64'
 --work string    Relative runner work directory (default _work)
 --replace        Replace any existing runner with the same name (default false)

[timharris777]

Not sure, but it seems to work just fine. :-)
./run.sh --once

[adiroiban]

True... this is a low level command $ ./bin/Runner.Listener run --once

Thanks!

[tabossert]

@timharris777 that does seem to work, but this would really only be useful when running runner as a docker container, which works until you try to use Github's container stuff which throws

Container feature is not supported when runner is already running inside container.

So seems while docker in docker works, the full runner options don't quite work

[Dids]

I run the runners in containers, but can also run run container actions. Just need two things: /var/run/docker.sock mounted and the correct permissions to access it (without sudo, iirc).

I also mount the working directory and cache paths "as is", eg. /tmp/ghrunner:/tmp/ghrunner, so they're the same paths on the host and inside any containers, including container actions.

[tabossert]

@Dids I get this when using the container actions

"Container feature is not supported when runner is already running inside container"

How were you able to get around this?

[neilhwatson]

True... this is a low level command $ ./bin/Runner.Listener run --once

We have VM based private runners. If this could make the runner process stop the condition could be used to make the VM shutdown causing VM autoscaling to kill node and make fresh one. Thus ephemeral. Has anyone tried this?

[neilhwatson]

We're calling this ephemeral runners and it's something we're looking at now. We're also looking at the service being cooperative in this scenario (service doesn't assign more work, runner exits, service auto cleans up registration).

If it's OK with you, I'll rebrand this ephemeral ...

Public runners already work this way. Why can't you offer up that code work on customer's Azure, AWS, or other cloud?

[j3parker]

@neilhwatson see #510 , it's been worked on in #660 .
This issue can probably be closed as a dupe

[tabossert]

@neilhwatson Yes I have scripted my runners in a way that terminates ec2 instances afterwards, and I always have docker runners that restart after a workflow is run. I don't use the service I just run the config.sh followed by ./run.sh --once

I believe this issue can be closed

[Nick-Goorwah]

Try the following:

1. On your Linux container: restart and re-authenticate via SSH to your container then re-run: ./run.sh

Azure, AWS and other virtual hosting services allow you to restart your container which kills any hanging process...