Feature: Reload ACL flags upon receiving a signal

[ibrahimmenem]

👋 great job here!
I'm interested in being able to reload ACLs without losing ongoing connections.
Given that this issue is implemented, after receiving a sighup, ghostunnel would reload the ACLs so for example, one can add/remove an organization without the need to change the command line or interrupt the current connections.

1. Would you accept a PR for this feature?
2. Any implementation guidelines?

Thanks

[mcpherrinm]

Note this is achievable today simply by re-executing a new ghostunnel and killing the old one. Since it sets SO_REUSEPORT by default, you can run multiple instances in parallel.

I think the most straightforward way to implement this would actually be a full configuration reload, with an internal re-execution version of just starting a new process. But maybe keeping some state (like metrics).

Either way this is definitely blocked on #252

[csstaub]

Agree, SO_REUSEPORT solves most of this. Unfortunately on Linux it's possible for a connection to be dropped even with SO_REUSEPORT because the accept queues are distinct for each process. See https://lwn.net/Articles/542866/. I believe it's theoretically possible to fix that by pushing an EBPF program onto the socket to prevent the accept/close race condition, but I've never actually tried that.

[csstaub]

Some useful links I found on this topic:
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/net/reuseport_bpf.c
https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-more-hacks/

[mcpherrinm]

huh the ebpf trick is neat; I hadn't seen that before.

If we do in-process reloads, theoretically at least that's not really a concern as we could dispatch in-process on a single listening queue. Not sure how much work that would be to build into Ghostunnel, though.

Or, if we keep the TLS/packet forwarding stack running, we could ensure that everywhere that uses config that supports reloading pulls from a config that's reloaded, ala the certloader atomic.LoadPointer stuff

[csstaub]

Yeah, in-process reloading would avoid this and probably what we'll end up doing I guess. Though I really like the (theoretical) simplicity of reloading the entire process, that way there never needs to be complicated logic or careful handling anywhere for reloading whatever parts of the config.

[ibrahimmenem]

Thank you both for the fast responses!

Though I really like the (theoretical) simplicity of reloading the entire process

If I were to maintain this project I'd push this option to the maximum :)

I guess at the short term (low traffic) the solution of re-executing a new ghostunnel would be good enough for us!

I'll keep an eye on this ticket to see how we can contribute.

[csstaub]

This is kinda possible now (on latest master) using OPA policies & reloading. You'll have to translate your flags into a policy file, but after that you can edit the policy file and have it reload the same way as certificates.

[ibrahimmenem]

We can close this issue if you want :) thanks Cedric

[csstaub]

Will do. Docs on OPA policies are here for anyone who's interested in this feature: https://github.com/ghostunnel/ghostunnel/blob/master/docs/ACCESS-FLAGS.md