Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
ksindi committed Jun 15, 2016
1 parent f941c76 commit dd5d256
Showing 1 changed file with 6 additions and 8 deletions.
14 changes: 6 additions & 8 deletions README.md
Expand Up @@ -6,8 +6,6 @@ define an S3 bucket.

## Install

We aim to get the project on PyPI but for now you can just run:

```
git clone https://github.com/jwplayer/sparksteps.git
cd sparksteps/
Expand Down Expand Up @@ -57,11 +55,11 @@ Prompt parameters:
```

The above example creates an EMR cluster of 1 node with default instance
type _m4.large_, uploads the pyspark script episodes.py to the specified
S3 bucket and copies the file from S3 to the cluster. Each operation is defined
as an EMR “step” that you can monitor in EMR. The final step is to run the spark
application with submit args that includes a custom spark-avro package and
app args “--input”.
type _m4.large_, uploads the pyspark script episodes.py and its dependencies to
the specified S3 bucket and copies the file from S3 to the cluster.
Each operation is defined as an EMR “step” that you can monitor in EMR. The
final step is to run the spark application with submit args that includes a
custom spark-avro package and app args “--input”.

## Run Spark Job on Existing Cluster

Expand All @@ -85,7 +83,7 @@ py.test sparksteps/tests.py

## Known Issues

If a conf file is specified, its parameters will overwrite anything specified
If a conf file is specified, its parameters will override anything specified
in the command line arguments.

## License
Expand Down

0 comments on commit dd5d256

Please sign in to comment.