Fetching contributors…
Cannot retrieve contributors at this time
128 lines (83 sloc) 3.5 KB
- Avoid deprecated DescribeJobFlows API
- Use DefaultAWSCredentialsProviderChain to discover AWS credentials
- EMR instance tags can be specified in cluster configuration
- Fix binary argument passthrough (e.g. "--no-bar")
- Fix some tests
- Hadoop streaming and step-jar selection enabled in local mode
- Project depends on aws-java-sdk 1.8.4
- add job-flow-role service-role as cli options
- Project depends on Clojure version 1.5.1
- Optional profiles can be enabled to use Clojure version 1.3 or 1.4
- Include CHANGES.txt and LICENSE in the tarball distribution
- Allow explicit action-on-failure setting for defstep
- New submit command to add a step to a running jobflow
- Move to lein2
- set the VisibleToAllUsers flag on all jobflows
- update to version 1.3.24 of the Amazon SDK for Java
- s3/cp now supports S3 directory to S3 directory copies
- Support in defcluster for two Amazon features
- set the hadoop-version with :hadoop-version which takes a string like "0.20.205"
- set supported-products with :supported-products, e.g.
(defcluster ...
:supported-products ["karmasphere-enterprise-utility"])
- In defstep, a value of "nil" should be converted to nil. This captures the common case
of "${foo}" when :foo is unset.
- New helper function lemur.core/eoval - use this in your defstep args to get the value
of an option.
E.g. (defstep ... (eoval foo))
This is better than "${foo}" in that it will return the value of :foo without converting
it into a string. This is most important for Boolean values, i.e. you want the value true,
not the string "true".
- Catch this common error and handle it as most people would expect:
(defstep ... "${foo}")
With no entry in (catch-args). Previously this resulted in a stackoverflow, as it repeatedly
repeatedly called itself to get the value of "${foo}". Now, the (catch-args) entry is
created implicitly with a default of nil.
- bugfix: When there is an explicit catch-args for a boolean option:
E.g. (catch-args [:foo? "some help"] false)
defstep would not recognize it, and would instead create a new option, non-boolean option :foo.
Now, you can create boolean options like this, which work with the catch-args example above:
(defstep ... (eoval :foo?))
- fix bootstrap-actions-test/mk-hadoop-config unit test
- Bugfix: :ami-version default should only be set in base.clj, not the command line defaults,
otherwise it is not over-ridable in a project specific base file.
- fire! accepts a "fn of eopts" which returns the cluster
- fire! accepts a "fn of eopts" for steps, which returns the list of Steps
- Error check for no jobdef
- Better diagnostic messages for diff-test-data
- fix clj1.3 regression for InstanceGroupConfig in emr.clj
- avoid non-fatal FileNotFound error for lein compile when is not available
- Added lein version command
- Include project.clj in tar (makes it easier to debug or play with the lemur code itself)
- Option to set availability-zone
- Clj 1.3 bug in
- Upgrade to Clojure 1.3.0
- documentation updates
- Add LICENSE and copyright notice
- Fix error when uploading 0-byte file to S3
- Add --[no-]show-progress option to enable[disable] ascii upload progress bars
- Look for explicit aws cred in the Environment: LEMUR_AWS_ACCESS_KEY, LEMUR_AWS_SECRET_KEY