Merge pull request #2015 from weaveworks/1-5-edits2

Reorganization of docs for WordPress

site/encryption/crypto-overview.md

@@ -0,0 +1,36 @@
+---
+title: Encryption and Weave Net
+layout: default
+---
+
+
+
+Weave can be configured to encrypt both the data passing over the TCP
+connections and the payloads of UDP packets sent between peers. This
+is accomplished using the [NaCl](http://nacl.cr.yp.to/) crypto
+libraries, employing Curve25519, XSalsa20 and Poly1305 to encrypt and
+authenticate messages. Weave protects against injection and replay
+attacks for traffic forwarded between peers.
+
+NaCl was selected because of its good reputation both in terms of
+selection and implementation of ciphers, but equally importantly, its
+clear APIs, good documentation and high-quality
+[go implementation](https://godoc.org/golang.org/x/crypto/nacl). It is
+quite difficult to use NaCl incorrectly. Contrast this with libraries
+such as OpenSSL where the library and its APIs are vast in size,
+poorly documented, and easily used wrongly.
+
+There are some similarities between Weave's crypto and
+[TLS](https://tools.ietf.org/html/rfc4346). Weave does not need to cater
+for multiple cipher suites, certificate exchange and other
+requirements emanating from X509, and a number of other features. This
+simplifies the protocol and implementation considerably. On the other
+hand, Weave needs to support UDP transports, and while there are
+extensions to TLS such as [DTLS](https://tools.ietf.org/html/rfc4347)
+which can operate over UDP, these are not widely implemented and
+deployed.
+
+**See Also**
+
+ * [How Weave Implements Encryption](/site/encryption/ephemeral-key.md)
+ * [Securing Containers Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md)

site/encryption/implementation.md

@@ -0,0 +1,185 @@
+---
+title: How Weave Implements Encryption
+layout: default
+---
+
+This section discusses the folloiwng topics: 
+
+ * [Establishing the Ephemeral Session Key](#ephemeral-key)
+ * [Key Generation](#csprng)
+ * [Encypting and Decrypting TCP Messages](#tcp)
+ * [Encypting and Decrypting UDP Messages](#udp)
+ * [Further Reading](#plugin)
+
+
+
+####<a name="ephemeral-key"></a>Establishing the Ephemeral Session Key
+
+For every connection between peers, a fresh public/private key pair is
+created at both ends, using NaCl's `GenerateKey` function. The public
+key portion is sent to the other end as part of the initial handshake
+performed over TCP. Peers that were started with a password do not
+continue with connection establishment unless they receive a public
+key from the remote peer. Thus either all peers in a weave network
+must be supplied with a password, or none.
+
+When a peer has received a public key from the remote peer, it uses
+this to form the ephemeral session key for this connection. The public
+key from the remote peer is combined with the private key for the
+local peer in the usual [Diffie-Hellman way](https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange), 
+resulting in both peersarriving at the same shared key. To this is appended the supplied
+password, and the result is hashed through SHA256, to form the final
+ephemeral session key. 
+
+The supplied password is never exchanged directly, and is thoroughly 
+mixed into the shared secret. Furthermore, the rate at which TCP connections 
+are accepted is limited by Weave to 10Hz, which thwarts online 
+dictionary attacks on reasonably strong passwords.
+
+The shared key formed by Diffie-Hellman is 256 bits long. Appending
+the password to this obviously makes it longer by an unknown amount,
+and the use of SHA256 reduces this back to 256 bits, to form the final
+ephemeral session key. This late combination with the password
+eliminates any "Man In The Middle" attacks: sniffing the public key
+exchange between the two peers and faking their responses will not
+grant an attacker knowledge of the password, and therefor, an attacker would
+not be able to form valid ephemeral session keys.
+
+The same ephemeral session key is used for both TCP and UDP traffic
+between two peers.
+
+###<a name="csprng"></a> Key Generation and The Linux CSPRNG
+
+Generating fresh keys for every connection
+provides forward secrecy at the cost of placing a demand on the Linux
+CSPRNG (accessed by `GenerateKey` via `/dev/urandom`) proportional to
+the number of inbound connection attempts. Weave has accept throttling
+to mitigate against denial of service attacks that seek to deplete the
+CSPRNG entropy pool, however even at the lower bound of ten requests
+per second, there may not be enough entropy gathered on a headless
+system to keep pace.
+
+Under such conditions, the consequences will be limited to slowing
+down processes reading from the blocking `/dev/random` device as the
+kernel waits for enough new entropy to be harvested. It is important
+to note that contrary to intuition, this low entropy state does not
+compromise the ongoing use of `/dev/urandom`. [Expert
+opinion](http://blog.cr.yp.to/20140205-entropy.html)
+asserts that as long as the CSPRNG is seeded with enough entropy (for example,
+256 bits) before random number generation commences, then the output is
+entirely safe for use as key material.
+
+By way of comparison, this is exactly how OpenSSL works - it reads 256
+bits of entropy at startup, and uses that to seed an internal CSPRNG,
+which is used to generate keys. While Weave could have taken
+the same approach and built a custom CSPRNG to work around the
+potential `/dev/random` blocking issue, the decision was made to rely
+on the [heavily scrutinised](http://eprint.iacr.org/2012/251.pdf) Linux random number
+generator as [advised
+here](http://cr.yp.to/highspeed/coolnacl-20120725.pdf) (page 10,
+'Centralizing randomness'). 
+
+>>**Note:**The aforementioned notwithstanding, if
+Weave's demand on `/dev/urandom` is causing you problems with blocking
+`/dev/random` reads, please get in touch with us - we'd love to hear
+about your use case.
+
+####<a name="tcp"></a>Encypting and Decrypting TCP Messages
+
+TCP connection are only used to exchange topology information between
+peers, via a message-based protocol. Encryption of each message is
+carried out by NaCl's `secretbox.Seal` function using the ephemeral
+session key and a nonce. The nonce contains the message sequence
+number, which is incremented for every message sent, and a bit
+indicating the polarity of the connection at the sender ('1' for
+outbound). The latter is required by the
+[NaCl Security Model](http://nacl.cr.yp.to/box.html) in order to
+ensure that the two ends of the connection do not use the same nonces.
+
+Decryption of a message at the receiver is carried out by NaCl's
+`secretbox.Open` function using the ephemeral session key and a
+nonce. The receiver maintains its own message sequence number, which
+it increments for every message it decrypted successfully. The nonce
+is constructed from that sequence number and the connection
+polarity. As a result the receiver will only be able to decrypt a
+message if it has the expected sequence number. This prevents replay
+attacks.
+
+####<a name="udp"></a>Encrypting and Decrypting UDP Packets
+
+UDP connections carry captured traffic between peers. For a UDP packet
+sent between peers that are using crypto, the encapsulation looks as
+follows:
+
+    +-----------------------------------+
+    | Name of sending peer              |
+    +-----------------------------------+
+    | Message Sequence No and flags     |
+    +-----------------------------------+
+    | NaCl SecretBox overheads          |
+    +-----------------------------------+ -+
+    | Frame 1: Name of capturing peer   |  |
+    +-----------------------------------+  | This section is encrypted
+    | Frame 1: Name of destination peer |  | using the ephemeral session
+    +-----------------------------------+  | key between the weave peers
+    | Frame 1: Captured payload length  |  | sending and receiving this
+    +-----------------------------------+  | packet.
+    | Frame 1: Captured payload         |  |
+    +-----------------------------------+  |
+    | Frame 2: Name of capturing peer   |  |
+    +-----------------------------------+  |
+    | Frame 2: Name of destination peer |  |
+    +-----------------------------------+  |
+    | Frame 2: Captured payload length  |  |
+    +-----------------------------------+  |
+    | Frame 2: Captured payload         |  |
+    +-----------------------------------+  |
+    |                ...                |  |
+    +-----------------------------------+  |
+    | Frame N: Name of capturing peer   |  |
+    +-----------------------------------+  |
+    | Frame N: Name of destination peer |  |
+    +-----------------------------------+  |
+    | Frame N: Captured payload length  |  |
+    +-----------------------------------+  |
+    | Frame N: Captured payload         |  |
+    +-----------------------------------+ -+
+
+This is very similar to the [non-crypto encapsulation](/site/router-topology/router-encapsulation.md).
+
+All of the frames on a connection are encrypted with the same
+ephemeral session key, and a nonce constructed from a message sequence
+number, flags and the connection polarity. This is very similar to the
+TCP encryption scheme, and encryption is again done with the NaCl
+`secretbox.Seal` function. The main difference is that the message
+sequence number and flags are transmitted as part of the message,
+unencrypted.
+
+The receiver uses the name of the sending peer to determine which
+ephemeral session key and local cryptographic state to use for
+decryption. Frames which are to be forwarded on to some further peer
+will be re-encrypted with the relevant ephemeral session keys for the
+onward connections. Thus all traffic is fully decrypted on every peer
+it passes through.
+
+Decryption is once again carried out by NaCl's `secretbox.Open`
+function using the ephemeral session key and nonce. The latter is
+constructed from the message sequence number and flags that appeared
+in the unencrypted portion of the received message, and the connection
+polarity.
+
+To guard against replay attacks, the receiver maintains some state in
+which it remembers the highest message sequence number seen. It could
+simply reject messages with lower sequence numbers, but that could
+result in excessive message loss when messages are re-ordered. The
+receiver therefore additionally maintains a set of received message
+sequence numbers in a window below the highest number seen, and only
+rejects messages with a sequence number below that window, or
+contained in the set. The window spans at least 2^20 message sequence
+numbers, and hence any re-ordering between the most recent ~1 million
+messages is handled without dropping messages.
+
+**See Also**
+
+ * [architecture documentation](https://github.com/weaveworks/weave/blob/master/docs/architecture.txt)
+ * [Securing Containers Across Untrusted Networks](/site/using-weave/security-untrusted-networks.md)

site/faq.md

@@ -0,0 +1,93 @@
+---
+title: Weave Net FAQ
+layout: default
+---
+
+
+
+###Q:How do I obtain the IP of a specific container when I'm using Weave?
+
+You can use `weave ps <container>` to see the allocated address of a container on a Weave network.  
+
+See [Troubleshooting Weave - List attached containers](/site/troubleshooting.md#list-attached-containers).
+
+
+###Q: How do I expose one of my containers to the outside world?
+
+Exposing a container to the outside world is described in [Exposing Services to the Outside](/site/using-weave/service-export.md).
+
+
+###Q: My dockerized app needs to check the request of a an application that uses a static IP. Is it possible to manually change the IP of a container?
+
+
+You can manually change the IP of a container using [Classless Inter-Domain Routing or CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). 
+
+For more information, refer to [Manually Specifying the IP Address of a Container](/site/using-weave/manual-ip-address.md). 
+
+
+###Q:Can I connect my existing 'legacy' network with a Weave container network?
+
+Yes you can. 
+
+For example, you have a Weave network that runs on hosts A, B, C. and you have an additional host, that we'll call P, where neither Weave nor Docker are running.  However, you need to connect from a process running on host P to a specific container running on host B on the Weave network.  Since the Weave network is completely separate from any network that P is connected to, you cannot connect the container using the container's IP address. 
+
+The simplest way to accomplish this would be to run Weave on the host and then run, `weave expose` to expose the network to any running containers.  See [Integrating a Network Host](/site/using-weave/host-network-integration.md) If this is not possible, you could also  expose a port on the container on host B and then connect to it. 
+
+You can read about exposing ports in [Service Exporting](/site/using-weave/service-export.md)
+
+
+###Q: Why am I seeing the same IP address assigned to two different containers on different hosts?
+
+Under normal circumstances, this should never happen, but it can occur if  `weave forget` and `weave rmpeer` was run on more than one host. 
+
+You cannot call `weave rmpeer` on more than one host. The address space, which was owned by the stale peer cannot be left dangling, and as a result it gets reassigned. In this instance, the address is reassigned to the peer on which `weave rmpeer` was run. Therefore, if you run `weave forget` and then `weave rmpeer` on more than one host at a time, it results in duplicate IPs on more than one host.
+
+Once the peers detect the inconsistency, they log the error and drop the connection that supplied the inconsistent data. The rest of the peers will carry on with their view of the world, but the network will not function correctly.
+
+Some peers may be able to communicate their claim to the others before they run `rmpeer` (i.e. it's a race), so what you can expect is a few cliques of peers that are still talking to each other, but repeatedly dropping attempted connections with peers in other cliques.
+
+For more information on see [Address Allocation with IP Address Management (IPAM)](/site/ipam/overview-init-ipam.md) and also, [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md)
+
+
+###Q: What is the best practise for resetting a node that goes out of service?
+
+When a node goes out of service, the best option is to call `weave rmpeer` on one host and then `weave forget` on all the other hosts.
+
+See [Starting, Stopping and Removing Peers](/site/ipam/stop-remove-peers-ipam.md) for an indepth discussion.
+
+
+###Q: What about Weave's performance? Are software defined network overlays just as fast as native networking?
+
+All virtualization techniques have some overhead, and Weave's overhead is typically around 2-3%. Unless your system is completely bottlenecked on the network, you won't notice this during normal operation. 
+
+Weave Net also automatically uses the fastest datapath between two hosts. When Weave Net can't use the fast datapath between two hosts, it falls back to the slower packet forwarding approach. Selecting the fastest forwarding approach is automatic, and is determined on a connection-by-connection basis. For example, a Weave network spanning two data centers might use fast datapath within the data centers, but not for the more constrained network link between them.
+
+For more information about fast datapath see [How Fast Datapath Works](/site/fastdp/fastdp-how-it-works.md)
+
+
+###Q:How can I tell if Weave is using fast datapath (fastdp) or not?
+
+To view whether Weave is using fastdp or not, you can run, `weave status connections`
+
+For more information on this command, see [Using Fast Datapath](/site/fastdp/using-fastdp.md)
+
+
+###Q: Does encryption work with fastdp?
+
+Encryption does not work with fast datapath. If you enable encryption using the `--password` option to launch Weave (or you use the `WEAVE_PASSWORD` environment variable), fast datapath will by default be disabled. 
+
+You can however have a mixture of fast datapath connections over trusted links, as well as, encrypted connections over untrusted links.
+
+See [Using Fast Datapath](/site/fastdp/using-fastdp.md) for more information
+
+###Q: Can I create multiple networks where containers can communicate on one network, but are isolated from containers on other networks?
+
+Yes, of course!  Weave allows you to run isolated networks and still allow open communications between individual containers from those isolated networks. You can find information on how to do this in [Application Isolation](/site/using-weave/application-isolation.md)
+
+
+**See Also**
+
+ * [Troubleshooting Weave](/site/troublehooting.md)
+ * [Troubleshooting IPAM](/site/ipam/troubleshooting.md)
+ * [Troubleshooting the Proxy](/site/weave-docker-api/using-proxy.md)
+

site/fastdp/fastdp-how-it-works.md

@@ -0,0 +1,30 @@
+---
+title: How Fast Datapath Works
+layout: default
+---
+
+
+Weave implements an overlay network between Docker hosts. Without fast datapath enabled, each packet is encapsulated in a tunnel protocol header and sent to the destination host, where the header is removed.  The Weave router is a user space process, which means that the packet follows a winding path in and out of the Linux kernel:
+
+![Weave Net Encapsulation](/images/weave-net-encap1-1024x459.png)
+
+
+The fast datapath in Weave uses the Linux kernel's [Open vSwitch datapath module](https://www.kernel.org/doc/Documentation/networking/openvswitch.txt). This module enables the Weave router to tell the kernel how to process packets:
+
+![Weave Net Encapsulation](/images/weave-net-fdp1-1024x454.png)
+
+Because Weave Net issues instructions directly to the kernel, context switches are decreased, and so by using `fast datapath` CPU overhead and latency is reduced. The packet goes straight from your application to the kernel, where the Virtual Extensible Lan (VXLAN) header is added (the NIC does this if it offers VXLAN acceleration). VXLAN is an IETF standard UDP-based tunneling protocol that enable you to use common networking tools like [Wireshark](https://www.wireshark.org/) to inspect the tunneled packets.
+
+![Weave Net Encapsulation](/images/weave-frame-encapsulation-178x300.png)
+
+Prior to version 1.2, Weave Net used a custom encapsulation format. Fast data path uses VXLAN, and like Weave Net's custom encapsulation format, VXLAN is UDP-based, and therefore needs no special configuration with network infrastructure. 
+
+>>Note:The required open vSwitch datapath (ODP) and VXLAN features are present in Linux kernel versions 3.12 and greater. If your kernel was built without the necessary modules Weave Net will fall back to the "user mode" packet path.
+
+
+**See Also**
+
+ * [Deploying Applications to Weave](/site/using-weave/deploying-applications.md)
+ * [Using Fast Datapath](/site/fastdp/using-fastdp.md)
+
+