This repository has been archived by the owner on Nov 30, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 798
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #4142 from krancour/isolate-etcd
docs(platform): document how to isolate etcd
- Loading branch information
Showing
5 changed files
with
99 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
The Deis Control Plane, Data Plane, and Router Mesh components all depend on an | ||
etcd cluster for service discovery and configuration. | ||
|
||
Whether built for evaluation or to host production applications, when managing a | ||
small Deis cluster (three to five nodes), it is reasonable to accept the | ||
platform's default behavior wherein etcd runs on every node within the cluster. | ||
|
||
In larger Deis clusters however, running etcd on every node can have a | ||
deleterious effect on overall cluster performance since it increases the time | ||
required for nodes to reach consensus on writes and leader elections. In such | ||
cases, it is beneficial to isolate etcd to a small, fixed number of nodes. All | ||
other nodes in the Deis cluster may run an etcd proxy. Proxies will forward | ||
read and write requests to active participants in the etcd cluster (leader or | ||
followers) without affecting the time required for etcd nodes to reach consensus | ||
on writes or leader elections. | ||
|
||
.. note:: | ||
|
||
The benefit of running an etcd proxy on any node not running a full etcd | ||
process is that any container or service depending on etcd can connect to | ||
etcd easily via ``localhost`` from any node in the Deis cluster. | ||
|
||
Also see `CoreOS cluster architecture documentation`_ for further details. | ||
|
||
.. _`CoreOS cluster architecture documentation`: https://coreos.com/os/docs/latest/cluster-architectures.html#production-cluster-with-central-services |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
:title: Isolating etcd | ||
:description: Configuring the cluster to isolate etcd | ||
|
||
.. _isolating-etcd: | ||
|
||
Isolating etcd | ||
============== | ||
|
||
.. include:: ../_includes/_isolating-etcd-description.rst | ||
|
||
.. note:: | ||
|
||
The approach documented here works as of Deis 1.9. Older versions of Deis | ||
utilize an older version of etcd that did not include the proxy | ||
functionality. | ||
|
||
cloud-config | ||
------------ | ||
|
||
To realize the topology described above, it is necessary, at the time of | ||
provisioning, to provide different cloud-config for those hosts that will run | ||
etcd and for those that will only run an etcd proxy. | ||
|
||
For the small, fixed number of hosts running full etcd and satisfying the | ||
"central services" role (as described in the CoreOS documentation), the | ||
cloud-config provided with Deis is sufficient. | ||
|
||
For hosts running only an etcd proxy, satisfying the "worker" role (as described | ||
in the CoreOS documentation), cloud-config must be tweaked slightly to include | ||
the ``-proxy on`` flag. For example: | ||
|
||
.. code-block:: yaml | ||
#cloud-config | ||
--- | ||
coreos: | ||
# ... | ||
- name: etcd.service | ||
command: start | ||
content: | | ||
# ... | ||
[Service] | ||
# ... | ||
ExecStart=/usr/bin/docker run --net=host --rm \ | ||
# ... | ||
-listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \ | ||
# ... | ||
--discovery <discovery url here> \ | ||
-proxy on | ||
# ... | ||
# ... | ||
# ... | ||
# ... | ||
Isolating etcd as described here requires subsets of a cluster's hosts to be | ||
configured differently from one another (including or excluding the | ||
``-proxy on`` flag). Deis provisioning scripts do not currently account for | ||
this, so managing separate cloud-config for each subset of nodes in the cluster | ||
is left as an exercise for the advanced operator. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters