Side project: Rewriting runner in Rust

[revolko]

Hey,

I am currently in my Rust phase, and I have decided to "rewrite" actions runner to Rust (with the twist that actions will be run in Docker container). However, I got stuck right at the beginning.

I have studied the current implementation and my understanding of runner registration is the following:

1. Get the runner token
2. Get the tenant token (based on the implementation, it is possible to get use_v2_flow but my requests have never got this set in responses).
3. Use runner token to make subsequent requests (either to list runners or to register custom runner).

I am struggling getting 3. step working. As I said, when getting the tenant token, I got a proper response, but never a flag with use_v2_flow set. I tried ignoring it and using v2_flow either way (as I could easily read what are the HTTP calls that needed to be executed), but that did not work. I either got 401 or 404 error (IMO both are caused by authentication issues).

I could not get anywhere using v2_flow so I decided to try and implement the VssConnection approach. Here I got stuck. I cannot figure out how to create VssConnection and honestly, I have no idea what it is. I have very little experience with .NET, but still tried to google something out, without success.

In the end, I have two questions:

1. Can I somehow force github API to allow me to use v2_flow?
2. If v2_flow is not possible, can anyone help me figuring out VssConnection?

P.S. My very simple implementation: https://github.com/revolko/github-docker-runner. I will be happy for any comments.

[ChristopherHX]

Hey,

Beware GitHub is moving their backend implementation step by step, I did something like this during 2021 in golang https://github.com/ChristopherHX/github-act-runner

1. Can I somehow force github API to allow me to use v2_flow?

I have the same problem, but since my project is from 2021 my vssconnection code is fully working.

No, I always need to wait for them to break my runner to test my changes based on the c# client codebase.

2. If v2_flow is not possible, can anyone help me figuring out VssConnection?

In this case I could help you.

Using a JIT token is a good idea, so you can skip registering your runner / use my project to generate a jit token that is not ephemeral.

The JIT token is a base64 encoded json with an json object containing files that are base64 encoded as well.

See here how I unpack that one https://github.com/ChristopherHX/github-act-runner/blob/3a9907019f05ef74f05e8c4e4fa2109ab4ebf809/runnerconfiguration/compat/actions_runner_compat.go#L199

How to parse this manually
https://github.com/ChristopherHX/github-act-runner/blob/3a9907019f05ef74f05e8c4e4fa2109ab4ebf809/runnerconfiguration/compat/actions_runner_compat.go#L117

See here how to get an access token for the runner listener

https://github.com/ChristopherHX/github-act-runner/blob/3a9907019f05ef74f05e8c4e4fa2109ab4ebf809/protocol/task_agent.go#L55

The rest is complex

• create an session
• get message
  • broker message
    • now you play ping pong with old / new services
    • You end up with a Real RunnerJobRequest Message

(You could skip this part and use my runner-worker plugin mechanism, so you just need to implement log and job status reporting)

This is not trivial

FYI My runner has a --trace flag, so you can spy what http requests are made and how they are answered by GitHub. The official runner is less verbose.

I decided to use golang, to not need to implement the whole execution logic from scratch like you are required to do

[revolko]

Thank you for reply. I will look into your project for sure.

Recently, I have realized that I can try and skip some validity calls (e.g. checking if given runner exists or not). At least for testing purposes. Because of that I got a bit further. I will share details later, if you are interested.

[ChristopherHX]

yeah, I also just POST to the runner creation endpoint, if it returns an error that the runner already exist I either ask for replacing or check the replace flag.

My code of the early days was able to brick the actions feature of a single repository, e.g. storing the public key of the wrong size (or with leading zeros in base64 etc.) inside the backend database. All database reads of that entity crashed most apis of the backend.

The most time consuming part has been waiting for the Actions Service stopping my abandoned Jobs, due to my initial code missed a jobcompleted call or just crashed before doing this. It is a good idea to store everything needed to complete a job on your disk so you can restart your process and completing orphan jobs to get the next one.