Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub server does not support configuring a self-hosted runner with 'DisableUpdate' flag. #112

Closed
laxgoalie392 opened this issue Oct 8, 2022 · 8 comments · Fixed by #121

Comments

@laxgoalie392
Copy link

Trying to use this library with my GHE resulted in the following error message:

The GitHub server does not support configuring a self-hosted runner with 'DisableUpdate' flag.

I'm using the codebuild provider and narrowed it down to the --disableupdate flag.

I'm new to this whole topic, but removing it from by buildspec manually resolved the issue. I'm guessing that the flag might even be irrelevant anyways since the runners are ephemeral?

What are the impacts of removing that flag? could we make it optional?

@kichik
Copy link
Member

kichik commented Oct 8, 2022

Since the runners are ephemeral, not disabling updates means it will check for updates and possibly apply them for every single job. That's quite a waste of resources. In Lambda runners it won't even work because the file system is read-only. We already have a job running weekly (interval is adjustable) that updates the runner image and version.

But if there is a setting that refuses that flag, we will have to make it optional.

@laxgoalie392
Copy link
Author

It seems to be refused by my company's GHE, unfortunately. i'll ask internally about it though.

for my understanding (and i'll try to do some more research on it later), when a runner picks up a job, it will perform some sort of update? and what is the weekly job that performs the update of the image and version? is it created by this CDK? is it a lambda running on a cron job that updates the image builder?

@kichik
Copy link
Member

kichik commented Oct 8, 2022

The runner will try to update itself without this flag for each job. The runner in this context is the GitHub .NET application that connects to GitHub, asks for a job, and executes it.

Our CDK construct creates a CloudWatch scheduled event that triggers a CodeBuild project that builds a new version of the runner image. That's a Docker image that contains the .NET application along with all of its requirements. So our CDK construct already takes care of updating the GitHub .NET runner application.

@laxgoalie392
Copy link
Author

That all makes sense to me i think.

do you want me to create a PR to add the default? if so, would you suggest adding it to the CodeBuildRunnerProps and then conditionally adding it to the codebuild spec?

@kichik
Copy link
Member

kichik commented Oct 8, 2022

All the providers will face the same issue, so this should probably go into RunnerProviderProps rather than just CodeBuildRunnerProps. Lambda actually copies the runner code to /tmp because it doesn't work in a read-only file system. So not using --disableupdates might work there too. But it's extra wasteful there as Lambdas are very time limited.

Since we probably going to have a disableRunnerUpdates argument that defaults to true, I was going to add a unit test to confirm it does the right thing. It's very easy in TypeScript to get default-true wrong.

If you want to create a PR for that, or even just get it started, that'd be very appreciated.

@kichik
Copy link
Member

kichik commented Oct 10, 2022

I dug in a little more and found that it checks for a new version even if --disableupdate is used. It doesn't update but it does check.

https://github.com/actions/runner/blob/408d6c579c36f0eb318acfdafdcbafc872696501/src/Runner.Worker/JobRunner.cs#L214

On my local machine updating was also not horribly slow. It took maybe 10 seconds. It also only updated when the runner was two major versions behind.

I am tempted to simplify this and only use --disableupdate if the user asked for a specific runner version for some reason. The image would be updated with the new version within at most a week. I doubt two major versions would come out in a week often.

@laxgoalie392
Copy link
Author

I haven't had much time to start making the changes this weekend. i've just had time here and there to validate your other fixes. I don't think i've ever seen someone maintaining a repo turn around my issues so fast! You are amazing!

If you are wanting to just bust this issue out too, go for it. i probably wont be able to start until tomorrow.

Side note: i was able to validate the docker changes from the other issue. I'm able to successfully run my job now!

@mergify mergify bot closed this as completed in #121 Oct 10, 2022
mergify bot pushed a commit that referenced this issue Oct 10, 2022
Some GitHub Enterprise instances forbid runners from using --disableupdate. We only truly need it when a user a chooses a specific runner version. When we update the runner image with its version every week, disabling updates shouldn't matter anyway. GitHub Actions seems to only auto-update the runner if we're two versions behind and that would be very rare in the span of a week (default image rebuild interval).

Originally we wanted to disable auto-updates because our runners are ephemeral and updating them for every single job would be wasteful. Since they only auto-update if we're two versions behind, I feel like we can drop this flag.

Fixes #112
Replaces #120
@kichik
Copy link
Member

kichik commented Oct 10, 2022

Thanks! 😁
It's my current passion project and it makes me happy to see bug reports come in because that's how I know people actually use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants