Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero-downtime deployment of Chisel #36

Open
abh1kg opened this issue Jun 23, 2017 · 6 comments
Open

Zero-downtime deployment of Chisel #36

abh1kg opened this issue Jun 23, 2017 · 6 comments

Comments

@abh1kg
Copy link

abh1kg commented Jun 23, 2017

First off, let me begin by saying that Chisel is a brilliant little project for accomplishing a lot of useful tasks. I personally use Chisel to tunnel SSH traffic to containers and VMs.

However, this got me thinking- how do you propose to achieve zero-downtime maintenance and deployment of the chisel server? If there are existing tunnels already created and active, would it be possible to switch the connections over to the new chisel server instance?

@jpillora
Copy link
Owner

Awesome :)

It's not possible to move existing connections into a new process. Each connection has significant state which couldn't be carried forward by a different process. chisel clients perform automatic reconnects on any disconnection so a deploy strategy of replacing the chisel server's binary and performing a restart would cause chisel clients to reconnect almost instantly.

However, this short chisel-to-chisel disconnection would also disconnect any user connections (e.g. SSH connections). I have thought of hiding any short chisel-to-chisel disconnections from the user, by buffering user data though this would result in complex edge cases, like, where a user program might think it has successfully transmitted data when it hasn't, and then this data is lost. Also, even if implemented, this would not work through a chisel upgrade.

Happy to hear of any other ideas

@vaijab
Copy link

vaijab commented Dec 12, 2017

@jpillora I assume running multiple chisel servers behind a TCP load balancer wouldn't work either, would it?

If hot-swapping chisel binary without bringing existing connections down is not an option, would you consider a feature that watches auth-file for changes? Just noticed that there is a feature request for this at #30

@jpillora
Copy link
Owner

jpillora commented Dec 13, 2017 via email

@abh1kg
Copy link
Author

abh1kg commented Dec 13, 2017

@jpillora Can CRIU work here with the tcp-connection-established flag? https://criu.org/Main_Page

@mjdilworth
Copy link

i dont think auth file changes is the ask. What is needed is a way to maintain network state between two or more processes. Going back many years, Cisco have done this with things like ASA firewalls and these were connected by fast ethernet. There is a way to do this, i just need to Google more. Or can someone point me in the right direction for the design pattern? Or are there any other open source projects that have implement the sane feature?

@timkeeler
Copy link

Would it be possible to use something like a Redis cluster to store/maintain state and share that across multiple Chisel instances? If so, building an generic option point state data to a url would be ideal - this way you can point it to various sources - Redis, MongoDB, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants