Setting up a private network

[AlexBHarley]

To me it doesn't look like this repository sets up a proper network. In the /network folder I see all the ansible and terraform work that has been done but from what I can see the nodes will never connect to each other.

So I run make validators-start or make fullnodes-start with the configured number of servers per az (availability zone) and number of az's... let's pretend two az's and one server in each.

How do these servers know about each other? I can't see anything editing the persistent_peers entry in the config.toml file.

[AlexBHarley]

According to this guide https://tendermint.com/docs/networks/terraform-and-ansible.html#ansible I do need to edit the config.toml or apply a --persistent-seeds flag

[alexanderbez]

This repository allows you to run a local network of 4 nodes via make localnet-start. As for spinning up networks in cloud environments, I'm not too sure of. Maybe @mircea-c or @greg-szabo can help there.

[AlexBHarley]

I've gotten something working by crawling the nodes and then using the /dial_peers endpoint.

[AlexBHarley]

Following up on this, are there any open source examples of peoples infrastructure?

Many things are missing in this repo which makes me think the problems we're solving wrt. infrastructure are being done by everyone.

In particular would be great to get some conversation going around setting up a sentry node architecture with multiple validator nodes and some kind of cloud KMS in place.

[alexanderbez]

Documentation does exist. I recommend taking a look at various validator's blogs/posts on their infra (e.g https://iqlusion.blog/a-look-inside-our-validator-architecture).

[gaia]

You can also get more information at:

https://forum.cosmos.network/t/sentry-node-architecture-overview/454
https://chainflow.io/cosmos-validator-architecture/
https://medium.com/figment-networks/full-disclosure-figments-cosmos-validator-infrastructure-3bc707283967
https://medium.com/stakewithus/our-validation-architecture-and-learnings-at-stakewith-us-1876470ec908
https://kb.certus.one/peers.html
https://medium.com/01node/on-validator-setup-part2-c78c05937927

Presently (I just checked again), Google, Amazon and Azure KMS still do not support Ed25519. So there is no cloud KMS yet for Cosmos. You'd have to go with the YubiHSM.