Visualisation of adea deploy algorithm
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
build
docs
server
try
www
.gitignore
README.md
bower.js
bower.json
deploy-demo.org
js-project.org
package.js
package.json

README.md

Deploy-demo

Playground and experiments for adea.

Containing:

  • Simulation of decentralized server deployment

Basically every server is supposed to self configure. So that means every server has to decide for themselves what other servers to connect to. The only info available (through serf) is a list of other servers. A server can also create network wide events and query/response other servers, but is not able to set state in the network. Any state is stored locally in CouchDB. These CouchDB instances will need to be linked up reliably with redundancy and robustness built-in. But the problem is that there is not going to be governing, central configuration server/agent/service. How to decide which other servers to link up with using CouchDB replications? The system will also have to self adjust to failing servers, failing replications etc.

Requirements are as few connections as possible per server and as few hops as possible from any server to any other server, but with guaranteed connectivity so it's similar to the degree-diameter problem in graph theory, which is not solved. But a good working solution is any small world network.

  • Watts-newman small network implementation

In the src directory is a nodejs script watts-newman.js that implements the algorithm by the same name. I wrote a basic depth first shortest path implementation for shortest path, but only roughly optimized it. It works, but is slow for V>30. I added a Dijkstra shortest path implementation found on the net. Much faster..

Using this script I can experiment and find the optimal values to use in the watts-newman small world network construction. See src/watts-newman.js for more details.

Run

bin/serve

and go to localhost:9001 for a simulation of a watts-newman small network.

It seems you can get a decent result with p=.8 and k=1. With p=0 you'll get the typical watts-newman ring.

Parameters need to be set in www/scripts/main.js for now.

All servers will need to run their own Adea agent. It is responsible for starting up docker services (haproxy, adea-ui, couchdb, logging, search and any other services, such as datatbases and apps), but also for setting up and maintaining couchdb replications, since this is where the adea agent will get its network state data from. The idea is that adea (any agent) can organise, configure, deploy, tear down docker containers, but also servers through the api's offered by the likes of Linode, Do, AWS etc.

If you took every adea server offline in the network but one, it should be able to rebuild itself autonomously. If there was no live version left, all you'd have to do is start up a adea agent on your own computer or on a VPS host, point it to a CouchDB instance, give it credentials and it would rebuild the network again, sharing load and responsibilities as soon as it had peers to work with.

For redundancy sake you'd have CouchDB instances hooked up to the network, such as offered by Cloudant, Iriscouch and Couchbase or any self-hosted CouchDB. To go even further, you could have the network data stored (encrypted) at free hosting services , preferably without using accounts, so adea can organise this itself. Then you could have database-less agents lurking at various servers and services perhaps that could restart the network. This is starting to sound like a virus, or bot network...

Install:

git clone git@github.com/michieljoris/deploy-demo

More ideas: http://xmpp.org/ irc? peer to peer, ala torrents vpn