Simulated Scaling Documentation #1383
Conversation
|
|
||
| You can remove nodes with `codenvy remove-node <ip>`. | ||
| Overlay networks require a distributed key-value store to be running on a node. We embed Consul, a key-value storage implementation as part of the Codenvy master node. We currently only support adding Linux nodes into an overlay network. |
There was a problem hiding this comment.
I think that key-value storage can not be on a docker node for workspaces. But I read these docs as it should be there. Can you rephrase that sentence.
| You can remove nodes with `codenvy remove-node <ip>`. | ||
| Overlay networks require a distributed key-value store to be running on a node. We embed Consul, a key-value storage implementation as part of the Codenvy master node. We currently only support adding Linux nodes into an overlay network. | ||
|
|
||
| The default network in Docker is a "bridge" network. If you know that your users will only ever have single container workspaces (this would be unusual and rare), then you can continue using the bridge network for scaling. Bridge networks can be scaled with Linux, Windows, or Mac. |
There was a problem hiding this comment.
We don't support more than 1 node with a bridge networking. So it is not scaling if user can't start new node on that network.
|
|
||
| 2: On the Codenvy master node, start Consul: `docker -H <CODENVY-IP>:2376 run -d -p 8500:8500 -h consul progrium/consul -server -bootstrap` | ||
|
|
||
| 3: On each workspace node, [configure and restart Docker](https://docs.docker.com/engine/admin/) with new options: `--cluster-store=consul://<CODENVY-IP>:8500`, `--cluster-advertise=<WS-IF>:2376`, and `--engine-insecure-registry=<CODENVY-IP>:5000`. |
There was a problem hiding this comment.
They have to add listening on TCP port into docker daemon conf also.
| 5: On the Codenvy master node, modify `codenvy.env` to uncomment or add: | ||
| ```json | ||
| # Comma-separated list of IP addresses for each workspace node | ||
| CODENVY_SWARM_NODES=<WS-IP>:2376,<WS2-IP>:2376,<WSn-IP>:2376 |
There was a problem hiding this comment.
Port depends on what is set in docker daemon conf. If we declare only 2376 here then we should require it is opened on a node. Or don't require 2376 and put here that port which is used on a node.
| CODENVY_SWARM_NODES=<WS-IP>:2376,<WS2-IP>:2376,<WSn-IP>:2376 | ||
| ``` | ||
|
|
||
| 6: Restart Codenvy with `codenvy/cli restart`. |
There was a problem hiding this comment.
Roman said that we can reload just swarm container. So you can test it with him. It is better way if it works.
There was a problem hiding this comment.
We do not have that syntax in the CLI. Until the CLI supports such a command, we only have restart.
There was a problem hiding this comment.
But then workspaces will be stopped. Not a good thing for scaling to stop whole application to just add 1 node
|
|
||
| This simulated scaling can be used for production, but it is generally discouraged because you would be running Docker in VMs that are on a host, and you are just taking on some extra I/O overhead that may not generally be necessary. However, this simulated-based approach gives good pointers on configuration of a distributed, cluster-based system if you were to use VMs-only. | ||
|
|
||
| As an example, the following sequence launches a 3-node cluster of Codenvy using Docker machine with a VirtualBox hypervisor. In this example, we launch 4 VMs: a Codenvy node, 2 additional workspace nodes, and a node to handle key-value storage. The key-value storage node is typically not part of the scaling configuration. However, Codenvy requires an "overlay" network, which is powered by a key-value storage provider such as Consule, etcd, or zookeeper. When running Codenvy on the host, we are able to setup an etcd key-value storage system automatically and associate the nodes with it. However, in a VM scale-out scenario, a dedicated key-value storage provider is needed. This particular example uses Consul key-value storage to setup the overlay network. |
There was a problem hiding this comment.
When running Codenvy on the host, we are able to setup an etcd key-value storage system automatically and associate the nodes with it
Is it about dockerized Codenvy or not? In dockerized Codenvy we don't install KV storage yet. Maybe we should not put that in docs for now?
If it is about non dockerized Codenvy I would recommend to specify that.
There was a problem hiding this comment.
It is only for dockerized codenvy. The simulated scaling example runs consul kv storage, so we are covered.
| # Grab the IP address of this VM and use it in other commands where we have <CODENVY-IP> | ||
| docker-machine create -d virtualbox --engine-env DOCKER_TLS=no --virtualbox-memory "2048" \ | ||
| --engine-opt host=tcp://0.0.0.0:2375 \ | ||
| --engine-opt="cluster-store=consul://<KV-IP>:8500" \ |
There was a problem hiding this comment.
I think that for codenvy master node which doesn't take part in workspace infrastructure we don't need cluster config
There was a problem hiding this comment.
You added the cluster configuration to your example. Can you update your example to indicate which of the 4 options are needed in the Docker daemon (on the Codenvy host) if the Codenvy master does not need to have workspaces?
There are four - which ones do we drop?
--cluster-store=consul://<CODENVY-IP>:8500, --cluster-advertise=<WS-IF>:2376, '--host=tcp://0.0.0.0:2375, and --engine-insecure-registry=:5000`
| --engine-opt="cluster-advertise=eth1:2376" codenvy | ||
|
|
||
| # Workspace Node 1 | ||
| docker-machine create -d virtualbox --engine-env DOCKER_TLS=no --virtualbox-memory "3000" \ |
There was a problem hiding this comment.
Maybe we should mention that 2048 is default WS size, so this node will be able to run single WS only. And we can use <3000 MB in that case.
| -v /home/docker/.codenvy:/data codenvy/cli:nightly init | ||
|
|
||
| # Setup Codenvy's configuration file to have the IP addresses of each workspace node | ||
| sed -i "s/^#CODENVY_WORKSPACE_AUTO_SNAPSHOT=true.*/CODENVY_WORKSPACE_AUTO_SNAPSHOT=true/g" ~/.codenvy/codenvy.env |
There was a problem hiding this comment.
Your example did not have sudo in the other docs - why is it needed now.
There was a problem hiding this comment.
I updated example yesterday with sudo
|
|
||
| # Start Codenvy with this configuration | ||
| docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock \ | ||
| -v /home/docker/.codenvy:/data codenvy/cli:nightly init |
There was a problem hiding this comment.
Use start instead of init
|
I've incorporated most comments - just need instructions about which parameters are not required on the master Daemon configuration if the master is not going to host workspaces. |
|
This issue is only about documenting how the systm works today. Other
issues are related to improving how the system operates, and this sort of
recycling is part of that.
Tyler Jewell | CEO | tyler@codenvy.com | 978.884.5355
…On Wed, Dec 21, 2016 at 10:26 AM, Alexander Garagatyi < ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In docs/_docs/admin-guide/admin-managing.md
<#1383>:
> +ifconfig
+```
+
+2: On the Codenvy master node, start Consul: `docker -H <CODENVY-IP>:2376 run -d -p 8500:8500 -h consul progrium/consul -server -bootstrap`
+
+3: On each workspace node, [configure and restart Docker](https://docs.docker.com/engine/admin/) with new options: `--cluster-store=consul://<CODENVY-IP>:8500`, `--cluster-advertise=<WS-IF>:2376`, and `--engine-insecure-registry=<CODENVY-IP>:5000`.
+
+4: Verify that Docker is running properly. Docker will not start if it is not able to connect to the key-value storage. TODO: ADD TEST INSTRUCTION. Each workspace node that successfully runs this command is part of the overlay network.
+
+5: On the Codenvy master node, modify `codenvy.env` to uncomment or add:
+```json
+# Comma-separated list of IP addresses for each workspace node
+CODENVY_SWARM_NODES=<WS-IP>:2376,<WS2-IP>:2376,<WSn-IP>:2376
+```
+
+6: Restart Codenvy with `codenvy/cli restart`.
But then workspaces will be stopped. Not a good thing for scaling to stop
whole application to just add 1 node
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#1383>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAX9Cu6wr09wkuAwCM40QBA4SvaETzhbks5rKW88gaJpZM4LR69g>
.
|
Adds a section to the managing doc of the admin-guide to discuss how to use Docker Machine to simulate running a 3 node Codenvy cluster where 2 of the nodes are workspace node and the third is a Codenvy node.
Need to get more feedback from @garagatyi on my last open comments, which is why we are having admins modify codenvy.env to setup the cluster instead of using the codenvy add-node command which is the prescribed way.