JobProxy is (should be) a proxy between an application that needs access to compute resources like starting a docker container or just starting a simple shell script and the various existing resource providing frameworks.
While developing an application that needs access to compute resources from a running Mesos, we were searching for a Mesos framework that fullfills our constraints (which were very simple at this point). However we found that there are exists more than one framework that can do the job, but which is right/best one for our application ? A proxy as abstraction layer between our application and the frameworks seems to be very helpful. The decision which framework to be used is then indepened from writting the application ...
- Java 8
Download the latest jar with the following command:
- RELEASE is an identifier you can find on release page i.e:
See JobProxyServerCLI for usage example.
List of implementing Frameworks:
- Mesos Chronos (https://mesos.github.io/chronos/) version 2.4.0
- DRMAA (https://www.drmaa.org/) using DRMAA V1.0 API
- JavaDocker (https://github.com/spotify/docker-client)
- Kubernetes (https://kubernetes.io)
Release/Development Branch Workflow
There are two branches one is the master branch with latest working version of JobProxy and the other one is the development branch for the next release.
On a release, the development branch must be merged into the master branch and the list of implementing Frameworks must be extended.
Each module has its own version number e.g.: https://github.com/jkrue/jobproxy/blob/master/JobProxyServer/pom.xml#L8
If the code is updated, the corresponding module version number must be updated.
If you update the classes in the JobProxyModel, please make sure to update the swagger specification, by running:
inside the JobProxyModel directory. This command will update the swagger.yaml inside the
Then please update the documentation (see below
How to extend our REST documentation?).
How to use jobProxy as a dependency?
Just go to this site and follow the instructions. Please keep in mind that you can import single subproject artifacts like JobProxyServer and not the whole JobProxy parent artifact.
How to build to build JobProxy?
Clone this project.
Compile this project with Maven.
mvn clean compile
How to package JobProxy?
Run the following command for packaging.
You will find the jar inside the
How to extend our REST documentation?
We are generating markup out of our swagger yaml by using the swagger2markup-cli tool. With the following command:
java -jar swagger2markup-cli-1.3.1.jar convert -c config.properties -i swagger.yaml -f api
doc.md is the output document
config.properties is the configuration file
How to build your own jobproxy client?
Just use the swagger code generator with our swagger.yaml to produce jobproxy client code in your favorite language.
The following call produces an java client from swagger specification:
swagger-codegen-cli.sh generate -i doc/REST/swagger.yaml -l java
How to version the project?
We decided that all modules should have the same version as the parent module. By using the below command in the project root you can update all child modules at once.
mvn versions:set -DnewVersion=<version>
- version = 2.1.0.alpha.2
Vagrant test environments
Vagrant is a handy tool which helps to setup "quasi"-real test environment for testing JobProxy's Modules.
Installing requirements for Vagrant environment
If you want to test JobProxy Modules you will need the following tools:
|Tool||Installation command for ubuntu|
|Vagrant||You can find package for Ubuntu/Debian here|
Starting the environment
After all prerequisites are done you should be able to start the environment.
since Vagrant is not able to start multiple machines in parallel due to instable behavior of VirtualBox (s. here ), the start via
$ vagrant up takes long time ans is therefore not used.
To start the Vagrant environment navigate to the subfolder containing the Vagrant file and run
This script lets start all machines sequentally without configuration and performs configuration in parallel after all machines are boot up.
$ cd JobProxyChronos/testing $ ./parallel_provision.sh
After the script is done you will get 1 Master node running ZooKepper and Chronos and 3 slave Mesos nodes. You'll also be able to access Mesos framework at http://10.0.0.2:5050 and Chronos Web GUI at http://10.0.0.2:8081