Skip to content
Autonomous service managing overlay networks with Docker integration
Go Dockerfile
Branch: master
Clone or download
Latest commit 7f6d0d5 Jun 10, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
api
client/sand Implement endpoint to get agent version and add version command on CLI Dec 7, 2018
cmd Tag v0.5.4 Dec 7, 2018
config Fix public hostname in dev Mar 20, 2019
dev-ssl First version with CLI and client package Feb 7, 2018
dist
endpoint Correctly deactivate endpoint even if the target namespace is not ava… Apr 30, 2019
etcd etcd.NewClient do not take a config.Config as argument anymore Nov 6, 2018
idmanager etcd.NewClient do not take a config.Config as argument anymore Nov 6, 2018
integrations/docker
ipallocator
netlink
netnsbuilder Correctly handle errors when piping connections Dec 6, 2018
netutils
network Fix endpoint deletion in overlay network in case of target namespace … Jun 10, 2019
store etcd.NewClient do not take a config.Config as argument anymore Nov 6, 2018
test/mocks
utils/tls
vendor Bump v0.4.5, graceful restart and fixing resource deallocating Nov 20, 2018
web Implement endpoint to get agent version and add version command on CLI Dec 7, 2018
.ackrc
.gitignore
.travis.yml
CHANGELOG.md Bump v0.5.7 Jun 10, 2019
Dockerfile Fix tests Nov 5, 2018
Gopkg.lock Bump v0.4.5, graceful restart and fixing resource deallocating Nov 20, 2018
Gopkg.toml Use go-utils etcd configuration helper Nov 5, 2018
LICENSE Create LICENSE Nov 5, 2018
README.md
Vagrantfile Rename networking-agent to sand Dec 28, 2017
docker-compose-vm1.yml Use go-utils etcd configuration helper Nov 5, 2018
docker-compose-vm2.yml
docker-compose.yml Merge pull request #32 from Scalingo/fix/31/deactive-endpoint May 2, 2019
magefile.go First version with CLI and client package Feb 7, 2018

README.md

Build Status

SAND Network Daemon

SAND is simple API designed to create overlay networks based on VXLAN in an infrastructure, basing its configuration on etcd.

The goal is to create private overlay network to link containers together, while being agnostic of the container technology. Libnetwork overlay network type is working but is so much bound to Docker, that you don't really have the choice of the container engine in your infrastructure and you get locked to Docker.

Design

The SAND network daemon should be installed on all the hosts which will have containers running in one of the overlay networks. All the created overlay networks can use the same IP range as they are completely isolated from each other. By default each overlay network will get IP address in 10.0.0.0/24

Creating a network is a no-op operation where a unique VXLAN ID is allocated and where the network configuration is stored on etcd.

When a first endpoint is added to a network, the service will create a dedicated network namespace containing the network VXLAN on the server adding the endpoint. A pair of veth interfaces will link the targeted namespace and the overlay namespace. All the veth interfaces are linked to the VXLAN with a bridge interface.

At the moment an endpoint is added or removed, all the other hosts having at least one endpoint in the same network are adding routes to the newly created endpoint modifying ARP and FDB tables of the VXLAN interface.

Installing

To install the server:

go get github.com/Scalingo/sand/cmd/sand-agent

To install the CLI:

go get github.com/Scalingo/sand/cmd/sand-agent-cli

Configuration (from environment)

  • NETNS_PATH default: /var/run/netns, location where SAND will create network namespace handlers
  • NETNS_PREFIX default: sc-ns-, name prefix for the network namespace handler files
  • HTTP_PORT default: 9999, port bind by the SAND HTTP API
  • PUBLIC_HOSTNAME default: $(hostname), endpoints are attached to a hostname, an agent won't accept to delete a endpoint if its not owned by its hostname
  • PUBLIC_IP IP of the host which will be used in the configuration of VXLAN routing rules
  • ROLLBAR_TOKEN If token is defined, all errors will be send to Rollbar
  • GO_ENV default: development, name of the environment, will be forwarded to Rollbar if configured

ETCD TLS configuration

Note that at least ETCD v3 is required to run SAND

  • ETCD_PREFIX default: /sc-net, configuration of SAND is stored in ETCD, this is the prefix used by the keys

  • ETCD_HOSTS default: http://127.0.0.1:2379, URL of the etcd instance/instances, ie. https://10.0.0.1:2379,10.0.0.2:2379,10.0.0.3:2379

  • ETCD_TLS_CERT Path to the client certificate to reach ETCD

  • ETCD_TLS_KEY Path to the private key authenticating the client certificate

  • ETCD_CACERT Path to the CA used by ETCD server certificate

HTTP TLS authentication

If all three are defined, server will serve HTTPS instead of HTTP with client certificate authentication will be enabled, refusing requests from unauthorized clients.

  • HTTP_TLS_CERT Path to the server certificate sent by the server
  • HTTP_TLS_KEY Path to the private key authenticating the server certificate
  • HTTP_TLS_CA Path to the CA used by SAND client certificates

Generating certificates

# Generate a CA valid for 5 years
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key -sha256 -days 1825 -out ca.pem

# Generate a client certificate valid for 2 years
openssl genrsa -out client.key 4096
openssl req -new -key client.key -out client.csr
openssl x509 -req -in client.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out client.pem -days 730 -sha256

References

GET requests accept parameters through URL query parameters POST requests accept a JSON body POST and GET requests retun a JSON body

  • GET /networks
  • POST /networks Parameters:
    • name - string - Name of the network, generated automatically if not set
    • ip_range - string - IP Range from which endpoint IP will be allocated from
  • DELETE /networks/{id}
  • GET /endpoints Parameters:
    • network_id - string - Filter the returned networks by network
    • hostname - string - Filter the returned endpoints by hostname
  • POST /endpoints Parameters:
    • network_id - string - ID to the network to use
    • ns_handle_path - string - path to the target namespace handler to inject the network
  • DELETE /endpoints/{id}

Go client package

Documentation: godoc

import "github.com/Scalingo/sand/client/sand

func main() {
	opts := []sand.Opt{
		sand.WithURL(a.config.ApiURL),
	}
	config, _ := sand.TlsConfig(
		caPath, certPath, keyPath,
	)
	opts = append(opts, sand.WithTlsConfig(config))
	client := sand.NewClient(opts)

	// Use the client
}

CLI

sand-agent-cli network-list
sand-agent-cli network-create [--name name]
sand-agent-cli network-delete --network id
sand-agent-cli endpoint-list [--network id] [--hostname hostname]
sand-agent-cli endpoint-create --network id --ns path_target_namespace_handler
sand-agent-cli endpoint-delete --endpoint id

Global flags

   --api-url value    when requests will be sent (default: "http://localhost:9999") [$SAND_API_URL]
   --cert-file value  identify HTTPS client using this SSL certificate file [$SAND_CERT_FILE]
   --key-file value   identify HTTPS client using this SSL key file [$SAND_KEY_FILE]
   --ca-file value    verify certificates of HTTPS-enabled servers using this CA bundle [$SAND_CA_FILE]
   --help, -h         show help
   --version, -v      print the version

Docker Integration

Start with the environment variable ENABLE_DOCKER_PLUGIN=true

It will use the port 9998 by default to communicate with Docker. Change DOCKER_PLUGIN_HTTP_PORT to customize it.

With Docker

On each server which should be part of a network the sand-id MUST be defined, as it should be common to all nodes running the network and docker is not returning the internal ID, so the knowledge has to be external from Docker.

Create the SAND network with:

$ sand-agent-cli network-create
New network created:
* id=320a669f-e465-4806-ab46-f2e6620c4311 name=net-sc-320a669f-e465-4806-ab46-f2e6620c4311 type=overlay ip-range=10.0.0.0/24, vni=4
$ SAND_ID="320a669f-e465-4806-ab46-f2e6620c4311"

Then create a Docker network:

$ docker network create --driver sand --ipam-opt sand-id=$SAND_ID --opt sand-id=$SAND_ID <name>

Finally, start as many containers as you want in the SAND network defined in the docker network:

$ docker run -it --rm --network <name> ubuntu:latest bash

Test in Development With Docker Compose

Create the SAND network with:

$ sand-agent-cli network-create
New network created:
* id=320a669f-e465-4806-ab46-f2e6620c4311 name=net-sc-320a669f-e465-4806-ab46-f2e6620c4311 type=overlay ip-range=10.0.0.0/24, vni=4

The id= part is important and is called SAND_ID in the remaining of the section.

Your docker-compose.yml file MUST use the version 2:

version: '2'
networks:
  sand-network:
    driver: sand
    driver_opts:
      sand-id: SAND_ID
    ipam:
      driver: sand
      options:
        sand-id: SAND_ID

services:
  service-1:
    ...
    networks:
      - sand-network

  service-2:
    ...
    networks:
      - sand-network

Testing

  • Single node with docker-compose, just run docker-compose up and you can start using SAND.

  • Multinodes using vagrant, just run vagrant up to start two nodes with Docker installed and SAND code mounted in them.

  • More tests and mocking of netlink commands

You can’t perform that action at this time.