Distributed Big Data Orchestration Service
Clone or download
mprimi and tgianos Make active jobs user limit value customizable on a per-user basis
The existing mechanism to limit the maximum number of jobs applies equally to all users.
This can result in having to configure its value to a 'known accepted worst case'. And that limit is then applied to everyone else.

The new mechanism allows to set a default value that applies to most users, plus a set of custom limits for individual users.
These limits can be higher or lower than the default.
Latest commit ad22a18 Dec 6, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
codequality Add protobuf/gRPC to build Jan 8, 2018
genie-agent-app Tweak message printed before the Agent starts up Dec 5, 2018
genie-agent Add a custom protocol resolver for S3 to the application context Nov 30, 2018
genie-app Fix `genie.users.activeLimit.*` properties for boots new bindings to … Jul 13, 2018
genie-client Apply automatic formatting to all Java files Jul 24, 2018
genie-common-external Gradle build file cleanup Nov 30, 2018
genie-common-internal Gradle build file cleanup Nov 30, 2018
genie-common Apply automatic formatting to all Java files Jul 24, 2018
genie-demo Update hadoop 2.7.1 download location for demo Sep 11, 2017
genie-docs Make active jobs user limit value customizable on a per-user basis Dec 6, 2018
genie-proto Gradle build file cleanup Nov 30, 2018
genie-security Apply automatic formatting to all Java files Jul 24, 2018
genie-test-web Set dummy Genie version in integration test properties Jul 24, 2018
genie-test Gradle build file cleanup Nov 30, 2018
genie-ui Updated npm-shrinkwrap file which updates format to lock version 1 Nov 30, 2018
genie-war Fix `genie.users.activeLimit.*` properties for boots new bindings to … Jul 13, 2018
genie-web Make active jobs user limit value customizable on a per-user basis Dec 6, 2018
gradle/wrapper Upgrade to Gradle 4.10.2 and Lombok 1.18.2 Oct 3, 2018
travis Fix references to in travis script to new correct Oct 4, 2018
.codecov.yml Add .codecov.yml for overriding default codecov behavior as necessary May 25, 2018
.editorconfig Add groovy files to the .editorconfig file Jul 11, 2018
.gitignore Breakout the UI from genie-web into genie-ui module Jun 21, 2018
.netflixoss Add keys for travis publish. Disable cloudbees jobs. Dec 15, 2015
.travis.yml Upgrade to git-publish plugin from gradle-git Oct 4, 2018
CHANGELOG.md Setup for documentation generation (#422) Oct 26, 2016
LICENSE Updating the build files with some clean up after the pull request fo… Feb 17, 2015
OSSMETADATA adding OSSMETADATA for NetflixOSS tracking Dec 11, 2015
README.md Enable publishing of code coverage data to codecov.io May 25, 2018
build.gradle Update spring-cloud-aws dependency to point to 2.0.3.BUILD-SNAPSHOT w… Nov 30, 2018
gradle.properties Upgrade to Spring Boot 2.0.7 Nov 30, 2018
gradlew Upgrade to Gradle 4 (#533) Jun 19, 2017
gradlew.bat Upgrade Gradle to 3.1 and Spring Platform to 2.0.8.RELEASE (#398) Sep 26, 2016
lombok.config Modifying DTO's for serializability and for using as keys and values … Feb 20, 2016
settings.gradle Break Boot FatJar generation into its own project Oct 11, 2018

README.md

Genie

Download License Issues NetflixOSS Lifecycle

Introduction

Genie is a federated job orchestration engine developed by Netflix. Genie provides REST-ful APIs to run a variety of big data jobs like Hadoop, Pig, Hive, Spark, Presto, Sqoop and more. It also provides APIs for managing the metadata of many distributed processing clusters and the commands and applications which run on them.

Documentation

See the official website to find documentation about Genie and specific documentation for various releases.

Demo

Genie has demo instructions available for all 3.x.x releases. Please see the release you're interested in demoing on the releases page. Click on the release and then demo docs.

Builds

Genie builds are run on Travis CI here.

Branch Build Coverage (coveralls.io) Coverage (codecov.io)
master Build Status Coverage Status codecov
3.3.x Build Status Coverage Status codecov
3.2.x Build Status Coverage Status codecov
3.1.x Build Status Coverage Status codecov
3.0.x Build Status Coverage Status codecov

Docker

Successful builds will also generate a docker image which is published to Docker Hub.

App Image

App Image

This is the image for the Spring Boot all in one jar. You can use docker pull netflixoss/genie-app:{version} to test the one you want.

You can run via docker run -t --rm -p 8080:8080 netflixoss/genie-app:{version}

WAR Image

WAR Image

This is the image that has Genie deployed as a WAR file within Tomcat. You can use docker pull netflixoss/genie-war:{version} to test the one you want.

You can run via docker run -t --rm -p 8080:8080 netflixoss/genie-war:{version}

Python Client

The Genie Python client has been moved into its own repo.