MConn is a framework to build custom service-discovery-solutions on top Mesosphere's Marathon
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docs
src/application
static/webserver
test
.bowerrc
.codoopts
.csslintrc
.dockerignore
.editorconfig
.gitignore
.travis.yml
CHANGELOG.md
Dockerfile
LICENSE
README.md
bower.json
coffeelint.json
gruntfile.coffee
package.json

README.md

Build Status Coffeedoc

MConn MConn

Notice:
This is a pre-release for friends and colleagues to check code styles, functions and usability. Currently, MConn undergoes an intensive development and testing. We will release our first version 0.1.0 in the next weeks. Please stay tuned!

--

MConn is an in Coffee-Script written HA Queue combined with a module management to work as an event-subscriber behind Mesosphere's Marathon to react on incoming task events and realize simple post processes (for Service-Discovery-Solutions) based on up/down scalings.

Workflows

One of our targets has been a framework design that provides numerous custom post processes without wasting time on the individuell application of the Marathon-Handling, the HA and the fault-tolerance.

Features

  • Web UI
  • JSON/Web API
  • Queue
  • API to react on incoming task events
  • Inventory sync. (timeeven, MConn leader election, Marathon leader election or by apicall) to work fault-tolerance
  • Module-Management
  • HA
  • Zookeeper as datastore for states and module presets
  • Leader/Non-Leader election
  • request proxy to the current leader

For Developers

MConn is for free and based on the Apache v2-License. Don’t hesitate to fork and customize it! The first documentation for configuring Mconn is available here.

Demo-Deployment

Requirements:

You have to assure that Mesos-DNS and each hostname can also be reached by Mesos-Slaves.

MConn's default enviroments for a fast deployment (inspired by DCOS) are:
Environment Variable
MCONN_HOST ${HOST}
MCONN_PORT ${PORT0}
MCONN_MODULE_PATH ${MESOS_SANDBOX}
MCONN_MARATHON_HOSTS leader.mesos:8080
MCONN_ZK_HOSTS leader.mesos:2181
MCONN_ZK_PATH ${MARATHON_APP_ID}

You can modify this with the following Marathon-Config:

"env": {
    "MCONN_MARATHON_HOSTS": "admin:password@10.11.12.13:8080,admin:password@10.11.12.14:8080",
    "MCONN_MARATHON_SSL": "true",
    "MCONN_ZK_HOSTS": "10.11.12.13:2181,10.11.12.14:2181,10.11.12.15:2181"
}

More informations about the MConn-Enviroments, see the docs.

Deployment scheme:

Workflows

1. Start MConn incl. the HelloWorld-Example:

The "uris"-paramenter in the following Marathon-Config will instruct Mesos to download the HelloWorkd-Example and extract it into the MESOS-SANDBOX:

curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" \
    leader.mesos:8080/v2/apps -d '{
    "id": "mconn",
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "livespotting/mconn:0.0.10",
            "network": "HOST"
        },
        "volumes": []
    },
    "ports": [31999],
    "cpus": 1,
    "mem": 1024,
    "instances": 3,
    "healthChecks": [
        {
            "path": "/v1/ping",
            "portIndex": 0,
            "protocol": "HTTP",
            "gracePeriodSeconds": 30,
            "intervalSeconds": 30,
            "timeoutSeconds": 30,
            "maxConsecutiveFailures": 3
        }
    ],
    "uris": [
        "https://github.com/livespotting/mconn-helloworld/archive/master.zip"
    ],
    "env": {
        "MCONN_MODULE_START": "mconn-helloworld-master"
    },
    "upgradeStrategy": {
        "minimumHealthCapacity": 0.5,
        "maximumOverCapacity": 0.0
    },
    "constraints": [
        [
            "hostname",
            "UNIQUE"
        ]
    ]
}'

After that call up the MConn UI at mconn.marathon.mesos:31999 (or get it from the Marathon UI)

MConn UI DemoApp

2. Add MConn as a Marathon Event-Subscriber:

To expand the "event_subscriber" on Marathon, start Marathon with "--event_subscriber http_callback" (more here) and afterwards install MConn as a HTTP-Endpoint:

curl -X POST leader.mesos:8080/v2/eventSubscriptions?callbackUrl=http://mconn.marathon.mesos:31999/v1/queue

3. Start a Demo-Application:

First you have to add a preset for the HelloWorld-Module instructing the modul what to do with our demo-app:

curl -H "Content-Type: application/json" -X POST -d '{"appId":"/bridged-webapp","moduleName":"HelloWorld","status":"enabled","options":{"actions":{"add":"Moin, Moin","remove":"Tschues"}}}' http://mconn.marathon.mesos:31999/v1/module/preset

The preset will generate a "Moin, Moin" if a task has got the status quo "TASK_RUNNING". It will generate a "Tschues", if the task is "TASK_FINISHED", "TASK_KILLED" or "TASK_FAILED".

MConn UI DemoApp

Afterwards start the Demo-App to see how the MConn-Queue-Management will work:

curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" \
    leader.mesos:8080/v2/apps -d '{
    "id": "bridged-webapp",
    "cmd": "python3 -m http.server 8080",
    "cpus": 0.5,
    "mem": 64,
    "instances": 4,
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "python:3",
            "network": "BRIDGE",
            "portMappings": [
                {
                    "containerPort": 8080,
                    "hostPort": 0,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 161,
                    "hostPort": 0,
                    "protocol": "udp"
                }
            ]
        }
    },
    "healthChecks": [
        {
            "protocol": "HTTP",
            "portIndex": 0,
            "path": "/",
            "gracePeriodSeconds": 5,
            "intervalSeconds": 20,
            "maxConsecutiveFailures": 3
        }
    ]
}'

MConn UI DemoApp

Or Mesos-Slave/Sandbox/Stdout:

[26-07-2015 18:36:35] [INFO] Task "bridged-webapp.8c6a13a9-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" created on queue (Module.HelloWorld)
[26-07-2015 18:36:35] [INFO] Task "bridged-webapp.8c6a3aba-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" created on queue (Module.HelloWorld)
[26-07-2015 18:36:35] [INFO] Starting worker for task "bridged-webapp.8c6a13a9-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" (Module.HelloWorld)
[26-07-2015 18:36:36] [INFO] Task "bridged-webapp.8c6a61cb-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" created on queue (Module.HelloWorld)
[26-07-2015 18:36:36] [INFO] Task "bridged-webapp.8c69ec98-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" created on queue (Module.HelloWorld)
[26-07-2015 18:36:36] [INFO] Moin, Moin "bridged-webapp.8c6a13a9-33b4-11e5-a556-2edad0a36318" (Module.HelloWorld)
[26-07-2015 18:36:36] [INFO] Task "bridged-webapp.8c6a13a9-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" state changed to "finished"! Task Queue is now "3" (Module.HelloWorld)
[26-07-2015 18:36:36] [INFO] Starting worker for task "bridged-webapp.8c6a3aba-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" (Module.HelloWorld)
[26-07-2015 18:36:37] [INFO] Moin, Moin "bridged-webapp.8c6a3aba-33b4-11e5-a556-2edad0a36318" (Module.HelloWorld)
[26-07-2015 18:36:37] [INFO] Task "bridged-webapp.8c6a3aba-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" state changed to "finished"! Task Queue is now "2" (Module.HelloWorld)
[26-07-2015 18:36:37] [INFO] Starting worker for task "bridged-webapp.8c6a61cb-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" (Module.HelloWorld)
[26-07-2015 18:36:38] [INFO] Moin, Moin "bridged-webapp.8c6a61cb-33b4-11e5-a556-2edad0a36318" (Module.HelloWorld)
[26-07-2015 18:36:38] [INFO] Task "bridged-webapp.8c6a61cb-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" state changed to "finished"! Task Queue is now "1" (Module.HelloWorld)
[26-07-2015 18:36:38] [INFO] Starting worker for task "bridged-webapp.8c69ec98-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" (Module.HelloWorld)
[26-07-2015 18:36:39] [INFO] Moin, Moin "bridged-webapp.8c69ec98-33b4-11e5-a556-2edad0a36318" (Module.HelloWorld)
[26-07-2015 18:36:39] [INFO] Task "bridged-webapp.8c69ec98-33b4-11e5-a556-2edad0a36318_TASK_RUNNING" state changed to "finished"! Task Queue is now "0" (Module.HelloWorld)

Author

License

(C) 2015 Livespotting Media GmbH

Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/