Permalink
Browse files

Docker Gradle Integration

Docker RM now is built via Gradle.

Pull Request:
  Closes #55

Author:
  Brandon Gulla brandon@brandongulla.com
  • Loading branch information...
Brandon Gulla darinj
Brandon Gulla authored and darinj committed Nov 18, 2015
1 parent 4bce035 commit e1db4d8b8cdb4fd1994be9097ab6e439e8f28cc0
Showing with 75 additions and 70 deletions.
  1. +15 −0 build.gradle
  2. +1 −1 docker/Dockerfile
  3. +7 −13 docker/README.md
  4. +0 −55 docker/ResourceManager.dockerfile
  5. +50 −0 docker/build.gradle
  6. +1 −1 docker/myriad-bin/create-user.sh
  7. +1 −0 settings.gradle
View
@@ -21,6 +21,21 @@ allprojects {
apply plugin: 'eclipse'
}
buildscript {
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath 'com.bmuschko:gradle-docker-plugin:2.6.1'
}
}
repositories {
mavenCentral()
}
idea {
project {
languageLevel = '1.7'
View
@@ -43,7 +43,7 @@ ADD myriad-bin/install-yarn.sh /install-yarn.sh
RUN sh /install-yarn.sh
# Copy over myriad libraries
ADD /libs/* /usr/local/hadoop/share/hadoop/yarn/lib/
ADD ./libs/* /usr/local/hadoop/share/hadoop/yarn/lib/
# Initialize hadoop confs with env vars
View
@@ -1,12 +1,14 @@
# Using Myriad with Docker #
Docker is the easiest way to from 0 to Yarn on Mesos within minutes.
## Building the Resource Manager Docker
`./build-myriad.sh` will run the gradle scripts from the root myriad folder and compile all necessary libraries.
Run the following command:
```./gradlew buildRMDocker```
This will build the ResourceManager from src and save the image as *mesos/myriad*.
#Configuration Guide#
In order for the ResourceManager to operate correctly, you will need to provide 2 configuration files:
In order for the ResourceManager to operate correctly, you will need to provide 2 configuration files. These files will need to mounted from a directory into */myriad-conf* within the docker container.
* [myriad-config-default.yml](https://github.com/mesos/myriad/blob/phase1/myriad-scheduler/src/main/resources/myriad-config-default.yml)
* modified [yarn-site.xml](https://github.com/mesos/myriad/blob/phase1/docs/myriad-dev.md)
@@ -17,19 +19,11 @@ In order for the ResourceManager to operate correctly, you will need to provide
```bash
docker run --net=host --name='myriad-resourcemanager' -t \
-v /path/to/configs:/myriad-conf \
-e HADOOP_NAMENODE="10.100.3.237:9000" \
mesos/myriad-resourcemanager
mesos/myriad
```
#Environment Variables#
* *HADOOP_NAMENODE* : *Required*
* *ALTERNATE_HADOOP_URL* : Optional - Allows user to override the hadoop distribution used by Myriad.
If you already had a working Vagrant instance, you will need to run `vagrant reload` in order to allow zookeeper and hdfs port-forwarding.
Still having problems connecting to Zookeeper? Be sure that your zkServer and MesosMaster values are correct in the [myriad-config-default.yml](https://github.com/mesos/myriad/blob/phase1/myriad-scheduler/src/main/resources/myriad-config-default.yml) file.
* *ALTERNATE_HADOOP_URL* : Optional - Allows user to override the hadoop distribution used by Myriad. This will download the *.tar.gz file to be used as the hadoop distribution of choice for Myriad.
---
<sub>

This file was deleted.

Oops, something went wrong.
View
@@ -0,0 +1,50 @@
task wrapper(type: Wrapper) {
gradleVersion = '2.9'
}
buildscript {
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath 'com.bmuschko:gradle-docker-plugin:2.6.1'
}
}
repositories {
mavenCentral()
}
apply plugin: 'com.bmuschko.docker-remote-api'
import com.bmuschko.gradle.docker.tasks.image.Dockerfile
import com.bmuschko.gradle.docker.tasks.image.DockerBuildImage
task copySchedulerJars(type:Copy){
def subdir = new File(project.buildDir, "libs")
if( !subdir.exists() ) { subdir.mkdirs() }
from '../myriad-scheduler/build/libs'
into 'libs'
}
task copyMyriadJars(dependsOn: ":myriad-scheduler:build") {
dependsOn(":myriad-executor:build")
dependsOn(copySchedulerJars)
}
task buildRMDocker(type: DockerBuildImage) {
dependsOn(copyMyriadJars)
if (System.env.DOCKER_HOST) {
url = "$System.env.DOCKER_HOST".replace("tcp","https")
if (System.env.DOCKER_CERT_PATH) {
certPath = new File(System.env.DOCKER_CERT_PATH)
}
}
else {
url = 'unix:///var/run/docker.sock'
}
inputDir = file('.')
tag = 'mesos/myriad'
}
@@ -27,7 +27,7 @@ export GROUP_UID=${GROUP_GID:='112'}
# Add hduser user
echo "Creating $HADOOP_USER user.."
groupadd $HADOOP_GROUP -g ${GROUP_UID}
useradd $HADOOP_USER -g $HADOOP_GROUP -u ${USER_UID} -s /bin/bash
useradd $HADOOP_USER -g $HADOOP_GROUP -u ${USER_UID} -s /bin/bash -d /home/${HADOOP_USER}
mkdir /home/${HADOOP_USER}
chown -R $HADOOP_USER:$HADOOP_GROUP /home/${HADOOP_USER}
View
@@ -29,3 +29,4 @@ rootProject.name = 'myriad'
include "myriad-commons"
include "myriad-scheduler"
include "myriad-executor"
include "docker"

0 comments on commit e1db4d8

Please sign in to comment.