Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic public IP handling #1072

Open
gufmar opened this issue Nov 4, 2019 · 17 comments
Open

Dynamic public IP handling #1072

gufmar opened this issue Nov 4, 2019 · 17 comments
Labels
enhancement New feature or request question Further information is requested

Comments

@gufmar
Copy link
Contributor

gufmar commented Nov 4, 2019

Some countries (eg Germany) have rules for ISPs to disconnect and change public assigned IP address once a day for broadband internet connections

cron jobs can watch the public IP

The feature request is to update (inject) the new public IP via jcli without restarting the node.

@mmahut
Copy link
Member

mmahut commented Nov 4, 2019

I think this should be system's operator duty to mediate such as specific case and should not build into the software directly.

@mzabaluev
Copy link
Contributor

mzabaluev commented Nov 5, 2019

By design (or at least by intent), restarting a node should not be a very dramatic event, both the node and the network should continue with minimal disruption. When the public IP changes, this means there is a time window when the node is unavailable, and it needs to reconnect to some peers and gossip about its new address to continue participation, so the process might as well restart as a systemd unit or whichever way the service is deployed that can be also used to provide it with an updated network configuration.

@NicolasDP NicolasDP added enhancement New feature or request question Further information is requested labels Nov 5, 2019
@gufmar
Copy link
Contributor Author

gufmar commented Nov 5, 2019

@mmahut to clarify: Yes definitively a sysop's duty to detect a dynamic changed IP (for those who want to run it in such an environment)
Based on such an event they need to decide if and how to react.
Current only possible option: rewrite the config and restart the node.

The feature request is to have a smooth, instant and smart way to update (inject) what effectively changed (the public gossiping IP) without restarting the whole node.

Today a restart is fast but will take more and more time with a growing DB. And with the current implementation, a restarted node has no idea and also no fresh (instant) information about the current epochs leader schedules.
So I wonder if it's possible - without any noteworthy node-internal complexity - to inject the new setting through jcli, similar to what is possible with jcli rest v0 leaders post -f?

@mmahut
Copy link
Member

mmahut commented Nov 5, 2019

@mmahut to clarify: Yes definitively a sysop's duty to detect a dynamic changed IP (for those who want to run it in such an environment)
Based on such an event they need to decide if and how to react.
Current only possible option: rewrite the config and restart the node.

Yes, I would suggest using systemd and BindsTo using sys-subsystem-net-devices, whenever the IP changes, systemd would rewrite the config line and restart the service.

Today a restart is fast but will take more and more time with a growing DB. And with the current implementation, a restarted node has no idea and also no fresh (instant) information about the current epochs leader schedules.

Good point, so I guess this issue is about implementing some kind of a configuration reload without restarting the entire node?

@mzabaluev has a valid point, that given the IP changed, your node is already down for the network itself.

@mzabaluev
Copy link
Contributor

Today a restart is fast but will take more and more time with a growing DB.

With a good journaling DB engine, it does not have to be so. sqlite that we use now is well tested (as opposed to how we use it), and in the future we may switch to something more lightweight, like TiKV, as a backend.

@rickymac68
Copy link

I agree with @gufmar we should not have to mess with public IPs and restarting nodes. I have run other crypto full nodes and never had to mess with my dynamic public IP which changes monthly. Automate as much as possible.

@mzabaluev
Copy link
Contributor

It should be possible to run a node without a public IP; in fact, this may be preferred for nodes behind firewalls/NATs. The only functionality disabled by not having a public IP address is being a trusted peer and having subscribers that connect to the node rather than the other way around. So for users without a statically mapped public IP address, a non-public node may be the preferred configuration until we implement NAT traversal automation and are sure that it works reliably.

Having said that, the network's strength depends on the number of publicly reachable nodes.

@mark-stopka
Copy link
Contributor

Already discussed in #839; automatic IP discovery, and NAT traversal should be implemented within the node itself. It's beneficial for the network to have many relay nodes, not only stake-pool nodes, it may very well be expected for those to run behind NAT with a UPnP enabled,...

@mmahut, the systemd workaround is like scratching left ear with a right arm if not worse; it's commonplace for p2p node to handle dynamic network topology changes (including a change of your own IP address).

@khwerhahn
Copy link

From a point of view of making it as easy as possible for less-techie personas to spin up a node and grow the network, it would definitely a beneficial feature. The less configuration there is for the average user, the better it is. If the next generation of Blockchain (Cardano) still has the same level of bad usability as the older Blockchains out there, it might not deter the developer but will deter the user side. It would fall into the "Delighter Feature" area from the Kano Model.

@JamesRobertKelley
Copy link

Will a node lose the slots won in the leader lottery if it is restarted?
Since uptime is so important we also need to consider server maintenance for security. The ability to change IP addresses could allow me to shutdown one node and fire up another with the same certificate to take over for the node in maintenance without losing slots. I don't know about those processes yet, so I could be missing something.

Another Issue is to add DNS resolution to the trusted nodes list. This could be part of a solution to this problem. When the IP address changes the node's DNS entry could be updated.

@mzabaluev
Copy link
Contributor

When the IP address changes the node's DNS entry could be updated.

That's even more prone to be applied with delay as DNS entries are usually cached.

@JamesRobertKelley
Copy link

When the IP address changes the node's DNS entry could be updated.

That's even more prone to be applied with delay as DNS entries are usually cached.

You can set the Time-to-Live on the DNS entry. I often lower it to 5 minutes and then 5 seconds before a server migration, which also offers a quick way to reverse the migration.

Round-robin DNS entries can also be used by setting two IP addresses with the same name. It is often used for load-balancing, but a client can automatically try the 2nd IP address if the first does not respond.

With a little more work Jormungandr could issue a "302 redirect" if it is not the current master for a certificate. The client would use DNS round-robin to pick up an IP, and if the wrong node was contacted a redirect message would point the client to the correct node. If the client does not receive a response it can try the other IP in the round-robin.

@mark-stopka
Copy link
Contributor

mark-stopka commented Nov 26, 2019 via email

@TheRealMrBojangles
Copy link

On a wireless LTE or 5G network you have 2 or 3 layers of dynamic IP's, at cell tower level, and at ISP level. These changes are quite frequent. You could be restarting the node every 2 min if you live in a high traffic area, with lots of mobile devices coming and going all the time...

@mark-stopka
Copy link
Contributor

mark-stopka commented Dec 22, 2019

@TheRealMrBojangles my LTE provider offers dynamic globally routable IPv4 that changes only upon link-reestablishment free of charge or fully static globally routable IPv4 for 4 EUR / month surcharge.

Don't extrapolate; every network is different. In networks behind CG-NAT you hardly going to configure any TCP port forwarding; which is also a situation Jormungandr should be able to deal with using other Jormungandr nodes as reflectors.

@TheRealMrBojangles
Copy link

@mark-stopka here in South Africa ISP's don't issue static Public IP on wireless networks only offered on fixed line fibre networks at an additional charge. Most areas don't have fibre connectivity. The whole of the African continent is pretty much on wireless. Our CBD runs on a copper network still, with fibre as a backup.

@bitnado
Copy link

bitnado commented Dec 29, 2019

is there any documentation on what happens when a jormungandr slot leader node re-connects to the network with a different ip address? is there any risk of being quarantined?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

9 participants