Open Service Broker is an implementation of the "Open Service Broker API" based on Spring Boot & Groovy. It enables platforms such as Cloud Foundry & Kubernetes to provision and manage services.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.circleci #143: Execute setup in docker directory Sep 20, 2018
.github Create issue_template.md Mar 24, 2018
broker Fix bug that prevented isKubernetesUpdateSuccessful to return true (#180 Oct 15, 2018
client Add Copyright Headers Jun 22, 2018
docker #143: Add sleep in wait shield wait loop Sep 20, 2018
gradle/wrapper #153: Update gradle and all dependencies to the newest releases Sep 19, 2018
img Initial commit May 23, 2017
model Add Copyright Headers Jun 22, 2018
scripts Remove newlines for github API release body Oct 3, 2018
.gitignore #143: Support automated functional test for shield Sep 20, 2018
.travis.yml #97: Add influxDB to travis by using docker service Aug 2, 2018
CHANGELOG.md Bump release version to 5.0.17 Oct 15, 2018
LICENSE Update LICENSE May 29, 2017
NOTICE Add NOTICE Jun 22, 2018
README.md update README Sep 20, 2018
THIRD-PARTY-NOTICES.txt Update THIRD-PARTY-NOTICES.txt Jun 22, 2018
build.gradle Remove space Sep 27, 2018
circleci-publish.sh #141: Fix logic error for uploading to sonatype Aug 20, 2018
gradle.properties Bump next develop version Oct 15, 2018
gradlew Initial commit May 23, 2017
gradlew.bat Initial commit May 23, 2017
manifest-template.yml Update README and manifest-template.yml with SPRING_APPLICATION_JSON Mar 19, 2018
openshift-deploy.sh openshift deployment script Apr 9, 2018
secring.gpg.enc Add publish to OSSRH Jul 10, 2017
settings.gradle module renaming May 30, 2017
travis-publish.sh Fix Gradle task to upload Snapshots for Broker module Jul 11, 2017

README.md

Open Service Broker

Status

Master Branch: CircleCI

Develop Branch: CircleCI

Introduction

Open Service Broker is an implementation of the Open Service Broker API. It enables platforms such as Cloud Foundry & Kubernetes to provision and manage services.

Open Service Broker is built in a modular way and one service broker can host multiple services.

Service broker offers extra functionality regarding Billing, Backup/Restore on top of the Open Service Broker API.

Services can be provisioned synchronously and/or asynchronously and the goal of this project is to provide a framework with which any service can easily be provisioned.

Service BrokerThe image above shows the high level architecture.

The following flow chart shows interactions for service provisioning and service binding for MongoDB Enterprise service.

flow chart

Development

Prerequisites

  • Java 1.8
  • MySQL / MariaDB Server

Deployment

Build

Build Service Broker using the gradlew script in the root directory of the repository.

$ ./gradlew clean build -x test -x integrationTest -x functionalTest -Ptomcat

The parameter called tomcat is for controlling if a tomcat runtime is integrated into the war.

Command below gives you a self executable jar

$ ./gradlew clean build -x test -x integrationTest -x functionalTest 

Database

To run the service broker locally a mariadb or mysql database with name CFbroker is required. The database tables will be generated automatically by the application. See the configuration section for more details.

Use Docker for MariaDB setup

docker run --name appc-cf-service-broker-db -e MYSQL_DATABASE=CFBroker -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -p 3306:3306 -d mariadb

Run

To run the built artifact, from the root directory the following commands can be executed:

java -jar -Dspring.config.additional-location=file:/some/path/servicebroker.yml broker/build/libs/service-broker-2.0.0-SNAPSHOT.war 

The config file passed can overwrite any default values. Providing an external config file is optional and when no external config file is provided, the default values are dictated by application.yml file(s).

or

./gradlew broker:bootRun

At the time being, Service Broker is recommended to run with only one instance to avoid concurrency issues. This issue will be fixed soon.

Openshift Deployment

For installing Service Broker on Openshift, use the openshift-deploy.sh script. This script creates a new MariaDB instance and provisions a new Service Broker instance & configures SB to use the newly created DB.

IDE/IntelliJ

When importing this project into IntelliJ, select the "create separate module per source set option". Intelli import

Cloud Foundry interactions

Follow the documentation to register the broker to Cloud Foundry.

Before a cf create-service-broker or update-service-broker call is made, please make sure that Service Broker is configured correctly. For configuring the catalog, see the service definition section.

Deploy in Cloud Foundry

It is also supported to host the open service broker as an app in cloudfoundry. The project contains in the root folder a 'manifest-template.yml' which contains a default configuration for the cloud foundry deployment. During the build the template file is being processed and a versioned manifest is created in the %rootdir%/broker/build/libs directory. To deploy:

  • build or ensure the Broker binaries are built
  • navigate to the libs folder for the project (where the manifest.yml is)
    • To override configurations, modify environment variable SPRING_APPLICATION_JSON in manifest.yml
  • login to cloud foundry with 'cf login ...'
  • execute the command 'cf push' which will parse the manifest and upload the application
    cf login -a https://api.scapp-console.swisscom.com --sso
    cf push

detailed documentation about manifest

JAVA_OPTS

JAVA_OPTS="$JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xmx2048m -XX:MaxPermSize=1024m -XX:+UseConcMarkSweepGC"

Configuration

The configuration file for the Service Broker is located under

broker/src/main/resources/application.yml

Service Definitions

Service definitions are maintained in the application.yml under the serviceDefinitions key. This key expects a list of service definitions as values. On startup, the service broker will validate that service definitions from in the database exist in the application.yml configurations based on GUID. Setting the value of serviceDefinitions to an empty list ([]), will bypass the verification.

Example Service Definition

Example Service Definition in application.yml configuration file.

serviceDefinitions:
- guid: 'udn9276f-hod4-5432-vw34-6c33d7359c12'
  name: 'mongodbent'
  description: 'MongoDB Enterprise HA v3.2.11'
  bindable: true
  asyncRequired: true
  internalName: 'mongoDbEnterprise'
  displayIndex: 1
  tags: []
  metadata:
    version: '3.2.11'
    displayName: 'MongoDB Enterprise'
  plans:
  - guid: 'jfkos87r-truz-4567-liop-dfrwscvbnmk6'
    name: 'replicaset'
    description: 'Replica Set with 3 data bearing nodes with 32 GB memory, 320 GB storage, unlimited concurrent connections'
    templateId: 'mongodbent-bosh-template'
    free: false
    displayIndex: 0
    containerParams:
    - template: ''
      name: 'plan'
      value: 'mongoent.small'
    - template: ''
      name: 'vm_instance_type'
      value: 'mongoent.small'
    metadata:
      storageCapacity: '320GB'
      memory: '32GB'
      nodes: '3'
      maximumConcurrentConnections: 'unlimited'
      dedicatedService: true
      highAvailability: true
      displayName: 'Small'

Updating/Creating Service Definitions over the service-definition API (defined below) requires the JSON format like following example.

{
    "guid": "udn9276f-hod4-5432-vw34-6c33d7359c12",
    "name": "mongodbent",
    "description": "MongoDB Enterprise HA v3.2.11",
    "bindable": true,
    "asyncRequired": true,
    "internalName": "mongoDbEnterprise",
    "displayIndex": 1,
    "tags": [],
    "metadata": {
      "version": "3.2.11",
      "displayName": "MongoDB Enterprise"
    },
    "plans": [
      {
        "guid": "jfkos87r-truz-4567-liop-dfrwscvbnmk6",
        "name": "replicaset",
        "description": "Replica Set with 3 data bearing nodes with 32 GB memory, 320 GB storage, unlimited concurrent connections",
        "templateId": "mongodbent-bosh-template",
        "free": false,
        "displayIndex": 0,
        "containerParams": [
          {
            "template": "",
            "name": "plan",
            "value": "mongoent.small"
          },
          {
            "template": "",
            "name": "vm_instance_type",
            "value": "mongoent.small"
          }
        ],
        "metadata": {
          "storageCapacity": "320GB",
          "memory": "32GB",
          "nodes": "3",
          "maximumConcurrentConnections": "unlimited",
          "dedicatedService": true,
          "highAvailability": true,
          "displayName": "Small"
        }
      }
    ]
}

Following endpoints to be deprecated

Get service definition

Via the example call below, service definitions for a given service id can be retrieved.

curl -u 'username:password' -X GET 'http://localhost:8080/custom/admin/service-definition/{service_id}'

Add service definition

Service Broker provides a way to update service definitions via HTTP calls.

Here is an example:

curl -u 'username:password' -X POST -H 'Content-Type: application/json' --data-binary '@path/to/definition/file.json' 'http://localhost:8080/custom/admin/service-definition'

This interface can be used for both adding a new service or updating an existing one. For an existing service, if a plan that is in use is tried to be removed an exception will be thrown.

Remove service definition

A service and its plan(s), which are not used i.e. which have no service instances, can be removed via a REST interface. Here is an example for how to delete a service that has the id service_id:

curl -u 'username:password' -X DELETE 'http://localhost:8080/custom/admin/service-definition/{service_id}'

Swagger

The Swagger API documentation can be accessed at http://localhost:8080/swagger-ui.html

Brokered Services

Bosh based services

Any bosh based service can be easily brokered. See here for Swisscom's bosh releases.
Check the class BoshBasedServiceProvider for details.

MongoDB Enterprise

https://github.com/swisscom/mongodb-enterprise-boshrelease

Kubernetes based services

Any Kubernetes based service can be provisioned with Open Service Broker. The asynchronous task is being created to prepare the provisioning of the service instance. Kubernetes Facade is using the client to execute a bunch of "templated" HTTP calls on Kubernetes API. All the templates can be read from the application.yml and are automatically read inside a specific ServiceProvider.

OpenWhisk

Open Service Broker can broker your local deployment of OpenWhisk.

Provision will create a new namespace.

An example of the provision json data (Parameters are optional).

cf create-service openwhisk basic my_open_whisk
 
cf create-service openwhisk basic my_open_whisk -c '{"namespace":"NAMESPACE"}'
{
    "service_id": "udn9276f-hod4-5432-vw34-6c33d7359c20",
    "plan_id": "jfkos87r-truz-4567-liop-dfrwscvbnm20",
    "parameters": {
        "namespace": "NAMESPACE"
    }
}

Binding will create a new subject within the namespace.

An example of the bind json data (Parameters are optional).

cf bind-service my_app my_open_whisk
 
cf bind-service my_app my_open_whisk -c '{"subject":"SUBJECT"}'
{
    "service_id": "udn9276f-hod4-5432-vw34-6c33d7359c20",
    "plan_id": "jfkos87r-truz-4567-liop-dfrwscvbnm20",
    "parameters": {
        "subject": "SUBJECT"
    }
}

Binding will return the HOST, UUID, and KEY. With these credentials, you can configure your local OpenWhisk CLI.

wsk property set --apihost HOST --auth UUID:KEY

As the OpenWhisk API does not provide the create/update/delete functionality that we were hoping for, we perform CRUD namespace/subject directly into the CouchDB.

Eden

Eden is a CLI to interact with any Open Service Broker API. All operations of the API are supported: discover/provision/bind/unbind/deprovision. After installing Eden you can use the following commands to interact with your broker:

export SB_BROKER_URL=http://localhost:8080 && export SB_BROKER_USERNAME=cc_admin && export SB_BROKER_PASSWORD=change_me
eden catalog
eden provision -s redis -p small -i test_redis

Before you are able to run these commands above make sure to have Open Service Broker running and desired service definitions got published.

Logging

The logging is based on Slf4j with the Log4J2 Extension. The log output can be modified with the default log4j2 configuration methods (e.g. log4j2.properties, log4j2.yaml ...). The default configuration returns a compacts json format with every item on a single line.

to allow local logging in human readable format the following files have been added to .gitIgnore: *log4j2-test

so that a file with log4j2-test.yaml can be added. To have a normal human readable output, use:

   status = error
   dest = err
   name = PropertiesConfig
   
   appender.console.type = Console
   appender.console.name = STDOUT
   appender.console.layout.type = PatternLayout
   appender.console.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
   
   rootLogger.level = info
   rootLogger.appenderRef.stdout.ref = STDOUT