CrateDB Mesos Framework

An integration framework that allows you to run and manage CrateDB via Apache Mesos.

This project is currently unmaintained. Contributions welcome. See below.

Note: this framework is experimental and not suitable for production use. Future changes in the API might break older installations!

Prerequisites

A JDK needs to be installed.

On OS X, we recommend using Oracle's Java. If you're using Linux, we recommend OpenJDK.

We recommend you use a recent Java 8 version.

Vagrant is also used.

Setup and Building

Clone the repository:

$ git clone https://github.com/crate/crate-mesos-framework.git

Build the JAR file:

$ ./gradlew fatJar

The JAR file can then be found in the build/libs/ directory.

This JAR file cannot be run directly as it requires a Mesos master instance and the Mesos native libraries.

Launching

This project ships with a Vagrantfile that can be used with Vagrant to launch virtual machines (VM) with Mesos installed.

Launch the VMs like so:

$ vagrant up

This will create and provision four VMs:

• mesos-master

  The Mesos master instance along with Zookeeper

• mesos-slave-{1..3}

  The Mesos slaves

If you have run vagrant up before, Vagrant boots the existing VMs.

Once the VMs are up-and-running, the CrateDB Mesos framework can be started inside the master VM. You can do that like so:

$ vagrant ssh -c "java -Djava.library.path=/usr/local/lib -jar /vagrant/build/libs/crate-mesos-*.jar --crate-version 0.54.9 --zookeeper 192.168.10.100:2181"

Inside the VM, /vagrant is mapped to the project root. This way, the JAR file can be accessed.

Hosts Entries

The static IPs of the Vagrant VMs are 192.168.10.100 for the master and 192.168.10.{101..103} for the slaves.

You can add them to your /etc/hosts file, like so:

192.168.10.100 mesos-master
192.168.10.101 mesos-slave-1
192.168.10.102 mesos-slave-2
192.168.10.103 mesos-slave-3

The Mesos WebUI should be available under http://mesos-master:5050 immediately after vagrant up is finished.

Once the CrateDB Mesos framework has been launched, the framework API becomes available at http://mesos-master:4040/cluster (if an API port is not otherwise specified).

Shortcut

You can re-build the JAR file and re-start the framework with this shortcut command:

$ bin/deploy --crate-version 0.47.7 --zookeeper 192.168.10.100:2181

Running via Marathon

One of the easiest ways to run CrateDB Mesos framework is via Marathon, on something like a DCOS cluster.

For installing Marathon, please refer to Mesosphere install guide. The Marathon WebUI should be available under http://mesos-master:8080 after setting things up.

Modify the template marathon/local.json file to suit your purposes and then submit the file to Marathon, like so:

$ curl -s -XPOST http://mesos-master:8080/v2/apps -d@marathon/local.json -H "Content-Type: application/json"

Once deployed, you can use the framework API to launch a CrateDB cluster. To do so, execute the resize command:

$ curl -sXPOST -H "Content-Type: application/json" <FRAMEWORK_HOST>:4040/cluster/resize -d '{"instances": <NUM_INSTANCES>}'

Here, <FRAMEWORK_HOST> is the hostname or IP of the host the framework is scheduled on, and <NUM_INSTANCES> is the desired number of CrateDB nodes.

Contributing

This project is primarily maintained by Crate.io, but we welcome community contributions!

See the developer docs and the contribution docs for more information.

Help

Looking for more help?

• Read the project docs
• Check out our support channels