runner: Ability to opt-out of using --ephemeral / opt-in to legacy --once for GHES older than 3.3

[mumoshu]

ARC would run with or without this patch, as long as you're using GitHub (Cloud) or you've already upgraded your GHES instance to 3.3 or greater.

Once this change has been merged and released as new runner images, the RUNNER_FEATURE_FLAG_EPHEMERAL envvar passed from the controller to the runner containers becomes noop, and whether to pass --ephemeral to the config.sh is determined solely by the presence of RUNNER_FEATURE_FLAG_EPHEMERAL envvar.

Ref #1196

Although we are going to eventually remove the ability to use the legacy --once flag as proposed in #1196, there might be folks still using legacy GHES versions 3.2 or earlier.

This commit removes the existing feature flag to opt-in for --ephemeral, while adding another feature flag RUNNER_FEATURE_FLAG_ONCE to opt-in for --once so that folks stuck in legacy GHES versions
can still use ARC.

Since this change every user starts using --ephemeral by default. If they see any issues on legacy GHES instance, RUNNER_FEATURE_FLAG_ONCE=true can be set to opt-in to keep using --once, which gives one more ARC release until they upgrade their GHES instance.

But beware, we won't support legacy GHES instances forever as it's going to be a maintenance nightmare. Please upgrade!

Ref #1196

[gjkim42]

This is a breaking change for the users who have used the RUNNER_FEATURE_FLAG_EPHEMERAL environment variable.
It might be better to notify there are a new env and the removed env to release notes.

(Do we have release notes for summerwind/runner images? I think we don't...)

[mumoshu]

@gjkim42 Hey. Since this change the --ephemeral flag is enabled by default. How can it be a breaking change? 🤔

[gjkim42]

Ah, I just found that the current actions-runner-controller does only support github enterprise >= 3.3.

I think our environment is broken because we have been using github enterprise 3.2 which does not support --ephemeral flag and we have been using summerwind/actions-runner:latest image.

[gjkim42]

Hey. Since this change the --ephemeral flag is enabled by default.

We've set RUNNER_FEATURE_FLAG_EPHEMERAL=false to disable --ephemeral flag but this PR makes the old environment variable useless.

[mumoshu]

@gjkim42 Thanks for clarifying. That makes sense. It's unfortunate that you temporarily broke your environment(it should be "temporarily" because you can still opt-in to use --once via RUNNER_FEATURE_FLAG_ONCE since this change, right?)

We've been advocating folks to upgrade GHES to 3.3+ since probably almost a year ago but apparently, it wasn't enough to make it 100% alright.

[gjkim42]

it should be "temporarily" because you can still opt-in to use --once via RUNNER_FEATURE_FLAG_ONCE since this change, right?

Yes, we can.

---

One casual suggestion:

Is there any plan to split summerwind/actions-runner from this repository? or to make official docker image in actions/runner?

Maybe we need to split summerwind/actions-runner part(which builds the container image of the actions-runner) from this repository so that we can manage the release cycle of the actions-runner.

[mumoshu]

Is there any plan to split summerwind/actions-runner from this repository? or to make official docker image in actions/runner?

Unfortunately no, almost entirely because of the lack of our resources.

We're a team of only two maintainers and I even wondered if we waited too long (almost a year) before making this change, being afraid if we wasted our human resources to maintain support for those legacy GHES versions no one is using anymore(which turned wrong just now).

Maybe we need to split summerwind/actions-runner part

Ideally yes, that would be a great idea!

[gjkim42]

I understood the points.
Anyway, thank you for all your great work. This project is awesome.

[toast-gear]

@gjkim42 to be clear this change was breaking on purpose (with the ability for end users to fix the problem themselves using the new env var). We've gone to great efforts to highlight --once is being removed but assumed there are probably people out there using it still. We decided to cause a sort of "brown out" as the last step in making people aware --once support is being removed (very very very soon).

See #1196 for long term ways of working around its removal if you are unable to get onto => GHES 3.3 however we would strongly recommend you upgrade as improvements to ARC will not be considering the GHES < 3.3 feature set or code base. If you do have a reason why keeping --once would be great (outside of you are running an out of date GHES instance) please do comment on #1196.