Skip to content

Commit

Permalink
Merge pull request #1158 from tskinn/add-dynamodb-provider
Browse files Browse the repository at this point in the history
add dynamodb backend
  • Loading branch information
emilevauge committed Mar 17, 2017
2 parents 2a61c90 + 72e35af commit 677899d
Show file tree
Hide file tree
Showing 25 changed files with 11,312 additions and 4 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@


Træfɪk (pronounced like [traffic](https://speak-ipa.bearbin.net/speak.cgi?speak=%CB%88tr%C3%A6f%C9%AAk)) is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease.
It supports several backends ([Docker](https://www.docker.com/), [Swarm](https://docs.docker.com/swarm), [Kubernetes](http://kubernetes.io), [Marathon](https://mesosphere.github.io/marathon/), [Mesos](https://github.com/apache/mesos), [Consul](https://www.consul.io/), [Etcd](https://coreos.com/etcd/), [Zookeeper](https://zookeeper.apache.org), [BoltDB](https://github.com/boltdb/bolt), [Eureka](https://github.com/Netflix/eureka), Rest API, file...) to manage its configuration automatically and dynamically.
It supports several backends ([Docker](https://www.docker.com/), [Swarm](https://docs.docker.com/swarm), [Kubernetes](http://kubernetes.io), [Marathon](https://mesosphere.github.io/marathon/), [Mesos](https://github.com/apache/mesos), [Consul](https://www.consul.io/), [Etcd](https://coreos.com/etcd/), [Zookeeper](https://zookeeper.apache.org), [BoltDB](https://github.com/boltdb/bolt), [Eureka](https://github.com/Netflix/eureka), [Amazon DynamoDB](https://aws.amazon.com/dynamodb/), Rest API, file...) to manage its configuration automatically and dynamically.

## Overview

Expand Down
9 changes: 9 additions & 0 deletions configuration.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ type GlobalConfiguration struct {
Eureka *provider.Eureka `description:"Enable Eureka backend"`
ECS *provider.ECS `description:"Enable ECS backend"`
Rancher *provider.Rancher `description:"Enable Rancher backend"`
DynamoDB *provider.DynamoDB `description:"Enable DynamoDB backend"`
}

// DefaultEntryPoints holds default entry points
Expand Down Expand Up @@ -406,6 +407,13 @@ func NewTraefikDefaultPointersConfiguration() *TraefikConfiguration {
defaultRancher.Watch = true
defaultRancher.ExposedByDefault = true

// default DynamoDB
var defaultDynamoDB provider.DynamoDB
defaultDynamoDB.Constraints = types.Constraints{}
defaultDynamoDB.RefreshSeconds = 15
defaultDynamoDB.TableName = "traefik"
defaultDynamoDB.Watch = true

defaultConfiguration := GlobalConfiguration{
Docker: &defaultDocker,
File: &defaultFile,
Expand All @@ -420,6 +428,7 @@ func NewTraefikDefaultPointersConfiguration() *TraefikConfiguration {
Mesos: &defaultMesos,
ECS: &defaultECS,
Rancher: &defaultRancher,
DynamoDB: &defaultDynamoDB,
Retry: &Retry{},
}

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@


Træfɪk is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease.
It supports several backends ([Docker](https://www.docker.com/), [Swarm](https://docs.docker.com/swarm), [Mesos/Marathon](https://mesosphere.github.io/marathon/), [Consul](https://www.consul.io/), [Etcd](https://coreos.com/etcd/), [Zookeeper](https://zookeeper.apache.org), [BoltDB](https://github.com/boltdb/bolt), [Amazon ECS](https://aws.amazon.com/ecs/), Rest API, file...) to manage its configuration automatically and dynamically.
It supports several backends ([Docker](https://www.docker.com/), [Swarm](https://docs.docker.com/swarm), [Mesos/Marathon](https://mesosphere.github.io/marathon/), [Consul](https://www.consul.io/), [Etcd](https://coreos.com/etcd/), [Zookeeper](https://zookeeper.apache.org), [BoltDB](https://github.com/boltdb/bolt), [Amazon ECS](https://aws.amazon.com/ecs/), [Amazon DynamoDB](https://aws.amazon.com/dynamodb/), Rest API, file...) to manage its configuration automatically and dynamically.

## Overview

Expand Down
71 changes: 71 additions & 0 deletions docs/toml.md
Original file line number Diff line number Diff line change
Expand Up @@ -1569,3 +1569,74 @@ Labels can be used on task containers to override default behaviour:
- `traefik.frontend.passHostHeader=true`: forward client `Host` header to the backend.
- `traefik.frontend.priority=10`: override default frontend priority
- `traefik.frontend.entryPoints=http,https`: assign this frontend to entry points `http` and `https`. Overrides `defaultEntryPoints`.
## DynamoDB backend
Træfɪk can be configured to use Amazon DynamoDB as a backend configuration:
```toml
################################################################
# DynamoDB configuration backend
################################################################
# Enable DynamoDB configuration backend
#
# Optional
#
[dynamodb]
# DyanmoDB Table Name
#
# Optional
#
TableName = "traefik"
# Enable watch DynamoDB changes
#
# Optional
#
Watch = true
# Polling interval (in seconds)
#
# Optional
#
RefreshSeconds = 15
# Region to use when connecting to AWS
#
# Required
#
# Region = "us-west-1"
# AccessKeyID to use when connecting to AWS
#
# Optional
#
# AccessKeyID = "abc"
# SecretAccessKey to use when connecting to AWS
#
# Optional
#
# SecretAccessKey = "123"
# Endpoint of local dynamodb instance for testing
#
# Optional
#
# Endpoint = "http://localhost:8080"
```
Items in the dynamodb table must have three attributes:
- 'id' : string
- The id is the primary key.
- 'name' : string
- The name is used as the name of the frontend or backend.
- 'frontend' or 'backend' : map
- This attribute's structure matches exactly the structure of a Frontend or Backend type in traefik. See types/types.go for details. The presence or absence of this attribute determines its type. So an item should never have both a 'frontend' and a 'backend' attribute.
8 changes: 6 additions & 2 deletions glide.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions glide.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ import:
- aws/endpoints
- aws/request
- aws/session
- service/dynamodb
- service/dynamodb/dynamodbiface
- service/dynamodbattribute
- service/ec2
- service/ecs
- package: cloud.google.com/go
Expand Down
181 changes: 181 additions & 0 deletions integration/dynamodb_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
package main

import (
"errors"
"io/ioutil"
"net/http"
"os"
"os/exec"
"strings"
"time"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/containous/traefik/integration/utils"
"github.com/containous/traefik/types"
"github.com/go-check/check"
checker "github.com/vdemeester/shakers"
)

type DynamoDBSuite struct {
BaseSuite
}

type DynamoDBItem struct {
ID string `dynamodbav:"id"`
Name string `dynamodbav:"name"`
}

type DynamoDBBackendItem struct {
DynamoDBItem
Backend types.Backend `dynamodbav:"backend"`
}

type DynamoDBFrontendItem struct {
DynamoDBItem
Frontend types.Frontend `dynamodbav:"frontend"`
}

func (s *DynamoDBSuite) SetUpSuite(c *check.C) {
s.createComposeProject(c, "dynamodb")
s.composeProject.Start(c)
dynamoURL := "http://" + s.composeProject.Container(c, "dynamo").NetworkSettings.IPAddress + ":8000"
config := &aws.Config{
Region: aws.String("us-east-1"),
Credentials: credentials.NewStaticCredentials("id", "secret", ""),
Endpoint: aws.String(dynamoURL),
}
var sess *session.Session
err := utils.Try(60*time.Second, func() error {
_, err := session.NewSession(config)
if err != nil {
return err
}
sess = session.New(config)
return nil
})
svc := dynamodb.New(sess)

// create dynamodb table
params := &dynamodb.CreateTableInput{
AttributeDefinitions: []*dynamodb.AttributeDefinition{
{
AttributeName: aws.String("id"),
AttributeType: aws.String("S"),
},
},
KeySchema: []*dynamodb.KeySchemaElement{
{
AttributeName: aws.String("id"),
KeyType: aws.String("HASH"),
},
},
ProvisionedThroughput: &dynamodb.ProvisionedThroughput{
ReadCapacityUnits: aws.Int64(1),
WriteCapacityUnits: aws.Int64(1),
},
TableName: aws.String("traefik"),
}
_, err = svc.CreateTable(params)
if err != nil {
c.Error(err)
return
}

// load config into dynamodb
whoami1 := "http://" + s.composeProject.Container(c, "whoami1").NetworkSettings.IPAddress + ":80"
whoami2 := "http://" + s.composeProject.Container(c, "whoami2").NetworkSettings.IPAddress + ":80"
whoami3 := "http://" + s.composeProject.Container(c, "whoami3").NetworkSettings.IPAddress + ":80"

backend := DynamoDBBackendItem{
Backend: types.Backend{
Servers: map[string]types.Server{
"whoami1": {
URL: whoami1,
},
"whoami2": {
URL: whoami2,
},
"whoami3": {
URL: whoami3,
},
},
},
DynamoDBItem: DynamoDBItem{
ID: "whoami_backend",
Name: "whoami",
},
}

frontend := DynamoDBFrontendItem{
Frontend: types.Frontend{
EntryPoints: []string{
"http",
},
Backend: "whoami",
Routes: map[string]types.Route{
"hostRule": {
Rule: "Host:test.traefik.io",
},
},
},
DynamoDBItem: DynamoDBItem{
ID: "whoami_frontend",
Name: "whoami",
},
}
backendAttributeValue, err := dynamodbattribute.MarshalMap(backend)
c.Assert(err, checker.IsNil)
frontendAttributeValue, err := dynamodbattribute.MarshalMap(frontend)
c.Assert(err, checker.IsNil)
putParams := &dynamodb.PutItemInput{
Item: backendAttributeValue,
TableName: aws.String("traefik"),
}
_, err = svc.PutItem(putParams)
c.Assert(err, checker.IsNil)

putParams = &dynamodb.PutItemInput{
Item: frontendAttributeValue,
TableName: aws.String("traefik"),
}
_, err = svc.PutItem(putParams)
c.Assert(err, checker.IsNil)
}

func (s *DynamoDBSuite) TestSimpleConfiguration(c *check.C) {
dynamoURL := "http://" + s.composeProject.Container(c, "dynamo").NetworkSettings.IPAddress + ":8000"
file := s.adaptFile(c, "fixtures/dynamodb/simple.toml", struct{ DynamoURL string }{dynamoURL})
defer os.Remove(file)
cmd := exec.Command(traefikBinary, "--configFile="+file)
err := cmd.Start()
c.Assert(err, checker.IsNil)
defer cmd.Process.Kill()
err = utils.TryRequest("http://127.0.0.1:8081/api/providers", 120*time.Second, func(res *http.Response) error {
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
if !strings.Contains(string(body), "Host:test.traefik.io") {
return errors.New("incorrect traefik config")
}
return nil
})
c.Assert(err, checker.IsNil)
client := &http.Client{}
req, err := http.NewRequest("GET", "http://127.0.0.1:8080", nil)
c.Assert(err, checker.IsNil)
req.Host = "test.traefik.io"
response, err := client.Do(req)
c.Assert(err, checker.IsNil)
c.Assert(response.StatusCode, checker.Equals, 200)
}

func (s *DynamoDBSuite) TearDownSuite(c *check.C) {
if s.composeProject != nil {
s.composeProject.Stop(c)
}
}
16 changes: 16 additions & 0 deletions integration/fixtures/dynamodb/simple.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
defaultEntryPoints = ["http"]

logLevel = "DEBUG"

[entryPoints]
[entryPoints.http]
address = ":8080"

[dynamodb]
AccessKeyID = "key"
SecretAccessKey = "secret"
Endpoint = "{{.DynamoURL}}"
Region = "us-east-1"

[web]
address = ":8081"
1 change: 1 addition & 0 deletions integration/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ func init() {
check.Suite(&MesosSuite{})
check.Suite(&EurekaSuite{})
check.Suite(&AcmeSuite{})
check.Suite(&DynamoDBSuite{})
}

var traefikBinary = "../dist/traefik"
Expand Down
16 changes: 16 additions & 0 deletions integration/resources/compose/dynamodb.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
dynamo:
image: deangiberson/aws-dynamodb-local
command: -sharedDb
ports:
- "8000:8000"
expose:
- "8000"

whoami1:
image: emilevauge/whoami

whoami2:
image: emilevauge/whoami

whoami3:
image: emilevauge/whoami

0 comments on commit 677899d

Please sign in to comment.