Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

make jobflows visible to other IAM users #16

Merged
merged 2 commits into from

3 participants

memory Marc Limotte tcc-jenkins
memory
Owner
  • set VisibleToAllUsers to true on all runjobflowrequests
  • update to latest version of the amazon sdk for java
tcc-jenkins tcc-jenkins make jobs visible to all IAM users
- set VisibleToAllUsers to true on all runjobflowrequests
- update to latest version of the amazon sdk for java
6e9920b
src/main/clj/com/climate/services/aws/emr.clj
@@ -259,6 +259,7 @@
(.setAmiVersion ami-version)
(.setSupportedProducts supported-products)
(.setBootstrapActions bootstrap-actions)
+ (.setVisibleToAllUsers true)
Marc Limotte Owner

Instead of hard-coding a value, this should be an optional key, like

(defn start-job-flow [name steps {:keys [log-uri bootstrap-actions ami-version supported-products visible-to-all-users]
:or {bootstrap-actions [] supported-products [] visible-to-all-users false}
...
(.setVisibleToAllUsers visible-to-all-users)

default should be false (compat with std EMR). We can change the default to true for TCC in our local codebase (in lemur_base.clj)

Also change com.lemur.core (around line 568) add this to the map:

:visible-to-all-users (true? (:visible-to-all-users evaluating-opts))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
memory memory merged commit fcd116f into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 13, 2012
  1. tcc-jenkins

    make jobs visible to all IAM users

    tcc-jenkins authored
    - set VisibleToAllUsers to true on all runjobflowrequests
    - update to latest version of the amazon sdk for java
  2. tcc-jenkins

    make visibletoallusers optional

    tcc-jenkins authored
    ...and off by default
This page is out of date. Refresh to see the latest.
5 CHANGES.txt
View
@@ -1,5 +1,10 @@
CHANGES
+1.2.1
+
+- set the VisibleToAllUsers flag on all jobflows
+- update to version 1.3.24 of the Amazon SDK for Java
+
1.2.0
- s3/cp now supports S3 directory to S3 directory copies
2  project.clj
View
@@ -17,7 +17,7 @@
; aws-java-sdk-1.3.3 does not specify the correct httpclient, so we do it explicitly
[org.apache.httpcomponents/httpclient "4.1.1"]
- [com.amazonaws/aws-java-sdk "1.3.3"
+ [com.amazonaws/aws-java-sdk "1.3.24"
:exclusions [javax.mail/mail org.apache.httpcomponents/httpclient]]
; TODO these two are only to support hipchat-- isolate that functionality, so these libs can be optional
5 src/main/clj/com/climate/services/aws/emr.clj
View
@@ -247,8 +247,8 @@
[jobflow-id steps]
(AddJobFlowStepsRequest. jobflow-id steps))
-(defn start-job-flow [name steps {:keys [log-uri bootstrap-actions ami-version supported-products]
- :or {bootstrap-actions [] supported-products []}
+(defn start-job-flow [name steps {:keys [log-uri bootstrap-actions ami-version supported-products visible-to-all-users]
+ :or {bootstrap-actions [] supported-products [] visible-to-all-users false}
:as all}]
(log/info (str "Starting JobFlow " all))
(let [instances (instances-config all)
@@ -259,6 +259,7 @@
(.setAmiVersion ami-version)
(.setSupportedProducts supported-products)
(.setBootstrapActions bootstrap-actions)
+ (.setVisibleToAllUsers visible-to-all-users)
(.setSteps steps))]
(.getJobFlowId (.runJobFlow *emr* request))))
3  src/main/clj/lemur/core.clj
View
@@ -561,7 +561,8 @@ calls launch - take action (upload files, start cluster, etc)
:keypair (:keypair evaluating-opts)
:ami-version (:ami-version evaluating-opts)
:hadoop-version (:hadoop-version evaluating-opts)
- :supported-products (:supported-products evaluating-opts)}
+ :supported-products (:supported-products evaluating-opts)
+ :visible-to-all-users (true? (:visible-to-all-users evaluating-opts))}
steps
(if (:enable-debugging? evaluating-opts)
(cons (emr/debug-step-config) steps)
Something went wrong with that request. Please try again.