Skip to content

Commit

Permalink
Update examples/spark for v1beta3
Browse files Browse the repository at this point in the history
  • Loading branch information
bcbroussard committed Apr 23, 2015
1 parent f9156c2 commit 32374ed
Show file tree
Hide file tree
Showing 4 changed files with 88 additions and 53 deletions.
18 changes: 9 additions & 9 deletions examples/spark/README.md
Expand Up @@ -29,19 +29,19 @@ instructions for your platform.
The Master service is the master (or head) service for a Spark
cluster.

Use the `examples/spark/spark-master.json` file to create a pod running
Use the `examples/spark/v1beta3/spark-master.json` file to create a pod running
the Master service.

```shell
$ kubectl create -f examples/spark/spark-master.json
$ kubectl create -f examples/spark/v1beta3/spark-master.json
```

Then, use the `examples/spark/spark-master-service.json` file to
Then, use the `examples/spark/v1beta3/spark-master-service.json` file to
create a logical service endpoint that Spark workers can use to access
the Master pod.

```shell
$ kubectl create -f examples/spark/spark-master-service.json
$ kubectl create -f examples/spark/v1beta3/spark-master-service.json
```

Ensure that the Master service is running and functional.
Expand Down Expand Up @@ -85,11 +85,11 @@ program.

The Spark workers need the Master service to be running.

Use the `examples/spark/spark-worker-controller.json` file to create a
Use the `examples/spark/v1beta3/spark-worker-controller.json` file to create a
ReplicationController that manages the worker pods.

```shell
$ kubectl create -f examples/spark/spark-worker-controller.json
$ kubectl create -f examples/spark/v1beta3/spark-worker-controller.json
```

### Check to see if the workers are running
Expand Down Expand Up @@ -164,10 +164,10 @@ SparkContext available as sc.
## tl;dr
```kubectl create -f spark-master.json```
```kubectl create -f v1beta3/spark-master.json```
```kubectl create -f spark-master-service.json```
```kubectl create -f v1beta3/spark-master-service.json```
Make sure the Master Pod is running (use: ```kubectl get pods```).
```kubectl create -f spark-worker-controller.json```
```kubectl create -f v1beta3/spark-worker-controller.json```
26 changes: 19 additions & 7 deletions examples/spark/spark-master-service.json
@@ -1,9 +1,21 @@
{
"id": "spark-master",
"kind": "Service",
"apiVersion": "v1beta1",
"port": 7077,
"containerPort": 7077,
"selector": { "name": "spark-master" },
"labels": { "name": "spark-master" }
}
"apiVersion": "v1beta3",
"metadata": {
"name": "spark-master",
"labels": {
"name": "spark-master"
}
},
"spec": {
"ports": [
{
"port": 7077,
"targetPort": 7077
}
],
"selector": {
"name": "spark-master"
}
}
}
38 changes: 23 additions & 15 deletions examples/spark/spark-master.json
@@ -1,20 +1,28 @@
{
"id": "spark-master",
"kind": "Pod",
"apiVersion": "v1beta1",
"desiredState": {
"manifest": {
"version": "v1beta1",
"id": "spark-master",
"containers": [{
"name": "spark-master",
"image": "mattf/spark-master",
"cpu": 100,
"ports": [{ "containerPort": 7077 }]
}]
"apiVersion": "v1beta3",
"metadata": {
"name": "spark-master",
"labels": {
"name": "spark-master"
}
},
"labels": {
"name": "spark-master"
"spec": {
"containers": [
{
"name": "spark-master",
"image": "mattf/spark-master",
"ports": [
{
"containerPort": 7077
}
],
"resources": {
"limits": {
"cpu": "100m"
}
}
}
]
}
}
}
59 changes: 37 additions & 22 deletions examples/spark/spark-worker-controller.json
@@ -1,28 +1,43 @@
{
"id": "spark-worker-controller",
"kind": "ReplicationController",
"apiVersion": "v1beta1",
"desiredState": {
"apiVersion": "v1beta3",
"metadata": {
"name": "spark-worker-controller",
"labels": {
"name": "spark-worker"
}
},
"spec": {
"replicas": 3,
"replicaSelector": {"name": "spark-worker"},
"podTemplate": {
"desiredState": {
"manifest": {
"version": "v1beta1",
"id": "spark-worker-controller",
"containers": [{
"name": "spark-worker",
"image": "mattf/spark-worker",
"cpu": 100,
"ports": [{"containerPort": 8888, "hostPort": 8888}]
}]
}
"selector": {
"name": "spark-worker"
},
"template": {
"metadata": {
"labels": {
"name": "spark-worker",
"uses": "spark-master"
}
},
"labels": {
"name": "spark-worker",
"uses": "spark-master"
"spec": {
"containers": [
{
"name": "spark-worker",
"image": "mattf/spark-worker",
"ports": [
{
"hostPort": 8888,
"containerPort": 8888
}
],
"resources": {
"limits": {
"cpu": "100m"
}
}
}
]
}
}
},
"labels": {"name": "spark-worker"}
}
}
}

0 comments on commit 32374ed

Please sign in to comment.