Moving automation into workbench libs #134

UrsaStutsman · 2018-01-24T20:30:46Z

Have you read CONTRIBUTING.md lately? If not, do that first.

I, the developer opening this PR, do solemnly pinky swear that:

I've documented my API changes in Swagger

In all cases:

Get two thumbsworth of review and PO signoff if necessary
Verify all tests go green
Squash and merge; you can delete your branch after this
Test this change deployed correctly and works on dev environment after deployment

jmthibault79 · 2018-01-25T16:22:51Z

automation/docker/application.conf.ctmpl

@@ -3,6 +3,7 @@
 {{with $environment := env "ENVIRONMENT"}}
 {{with $dir := env "ROOT_DIR"}}
 {{with $local_ui := env "LOCAL_UI"}}
+{{with $trialBillingPemSecrets := vault (printf "secret/dsde/firecloud/%s/common/trial-billing-account.pem" $environment)}}


Most of these additions are not used by Leonardo automation, but they are required by the current iteration of workbench-libs automation config. Removing these is a TODO but let's not block on it.

should vault-y confs be living in a public repo?

every repo will be getting this

jmthibault79 · 2018-01-25T16:24:33Z

automation/project/Dependencies.scala

@@ -35,13 +37,14 @@ object Dependencies {
    "com.typesafe.akka"   %%  "akka-testkit"        % akkaV     % "test",
    "com.typesafe.akka"   %%  "akka-slf4j"          % akkaV,
    "org.specs2"          %%  "specs2-core"   % "3.7"  % "test",
-    "org.scalatest"       %%  "scalatest"     % "2.2.6"   % "test",
+    "org.scalatest"       %%  "scalatest"     % "3.0.1"   % "test",


This update to match workbench-service-test required changes in how we use eventually

jmthibault79 · 2018-01-25T16:26:41Z

automation/src/test/scala/org/broadinstitute/dsde/firecloud/page/FireCloudView.scala

@@ -1,6 +1,6 @@
 package org.broadinstitute.dsde.firecloud.page

-import org.broadinstitute.dsde.workbench.util.WebBrowserUtil


TODO non-blocker: Remove most of the page classes from Leonardo

can we github ticket these todos?

yep I'll write something up

jmthibault79 · 2018-01-25T16:30:45Z

automation/src/test/scala/org/broadinstitute/dsde/workbench/leonardo/LeonardoTestUtils.scala

    val actualCluster = eventually {
      clusterCheck(Leonardo.cluster.get(googleProject, clusterName), clusterRequest.labels, googleProject, clusterName, Seq(ClusterStatus.Running, ClusterStatus.Error), clusterRequest.jupyterExtensionUri)
-    } (clusterPatience)


ScalaTest 3 makes it more difficult to pass patience as a parameter

jmthibault79 · 2018-01-25T16:32:24Z

docker/build.sh

-        GIT_SHA=$(git rev-parse ${GIT_BRANCH})
-        echo GIT_SHA=$GIT_SHA > env.properties  # for jenkins jobs
-        docker build -t $REPO:${GIT_SHA:0:12} .
+        echo "building $PROJECT docker image..."


These changes enable new-style Jenkins builds

jmthibault79 · 2018-01-25T16:53:33Z

jenkins retest

jmthibault79 · 2018-01-25T17:13:21Z

Initial Jenkins-pipeline failed because clusters were in Error state. Trying again.

jmthibault79 · 2018-01-25T18:23:06Z

jenkins retest

rtitle · 2018-01-25T18:35:04Z

automation/project/Dependencies.scala

@@ -7,11 +7,13 @@ object Dependencies {

  val workbenchModelV   = "0.8-d97f551"
  val workbenchGoogleV  = "0.10-b95b2c1"
+  val serviceTestV = "0.1-c5a3162-SNAP"


Should update to non-SNAP; I guess that is dependent on the workbench-libs PR being merged.

rtitle · 2018-01-25T18:36:33Z

automation/project/Dependencies.scala


  val excludeWorkbenchModel =   ExclusionRule(organization = "org.broadinstitute.dsde.workbench", name = "workbench-model_2.11")

  val workbenchModel: ModuleID =  "org.broadinstitute.dsde.workbench" %% "workbench-model"  % workbenchModelV
  val workbenchGoogle: ModuleID = "org.broadinstitute.dsde.workbench" %% "workbench-google" % workbenchGoogleV excludeAll excludeWorkbenchModel
+  val workbenchServiceTest: ModuleID = "org.broadinstitute.dsde.workbench" %% "workbench-service-test" % serviceTestV % "test" classifier "tests" excludeAll excludeWorkbenchModel


Wonder if we should be excluding both workbenchModel and workbenchGoogle?

rtitle · 2018-01-25T18:41:33Z

automation/src/test/scala/org/broadinstitute/dsde/workbench/config/LeoAuthToken.scala

+
+case class LeoAuthToken(value: String)
+
+case object LeoAuthToken extends AuthToken {


I am curious how this class is different from AuthToken, UserAuthToken, ServiceAccountAuthToken classes in workbench-libs:

https://github.com/broadinstitute/workbench-libs/tree/2801-sam/serviceTest/src/test/scala/org/broadinstitute/dsde/workbench/auth

Could one of those classes be used instead?

We've got a card already for the next iteration of this - I can add refactoring this to it

And I've added it.

rtitle · 2018-01-25T18:45:15Z

automation/src/test/scala/org/broadinstitute/dsde/workbench/leonardo/LeonardoTestUtils.scala

-  val ronAuthToken = AuthToken(LeonardoConfig.Users.ron)
-  val hermioneAuthToken = AuthToken(LeonardoConfig.Users.hermione)
-  val ronEmail = LeonardoConfig.Users.ron.email
+  // Ron and Hermione are on the dev Leo whitelist, and Hermione is a Project Owner


What whitelist? :)

Heh, good point. I would like to defer those changes please :)

rtitle

A couple minor questions, overall big 👍

aweng98

A very long sbt module name for workbenchServiceTest. project compiled cleanly, and Leonardo tests passed.

* use workbench libs * auth token for leo, other deletions, tweaks * adding methods to configs * adding some more info * forgot to render users.json * build/push swat tests docker image in jenkins build script * all up to date * syncing users json * Add campaignManagers to users * workbench-libs needs these (TODO remove) * fix for app conf * real fix for app conf * Hermione and Ron * more app conf fix * more app conf ctmpl * Update scalatest version for workbench-libs compatibility * implicits are the devil * dammit weasley * MORE PATIENCE * fix build script when pushing to develop * save git SHA during build script

UrsaStutsman and others added 19 commits January 24, 2018 15:01

use workbench libs

3ac67c2

auth token for leo, other deletions, tweaks

5bf19e0

adding methods to configs

da251fe

adding some more info

e130520

forgot to render users.json

5e02a3f

build/push swat tests docker image in jenkins build script

00ba59f

all up to date

97bbe73

syncing users json

2ac9569

Add campaignManagers to users

8876388

workbench-libs needs these (TODO remove)

055d8ea

fix for app conf

5e56838

real fix for app conf

f542d28

more app conf fix

b50be6b

Hermione and Ron

5d8447c

more app conf ctmpl

8f0515f

Update scalatest version for workbench-libs compatibility

11ba2d2

implicits are the devil

3d9573a

dammit weasley

ade7c3e

MORE PATIENCE

3c01fb9

UrsaStutsman changed the title ~~2801 sam redux~~ Moving automation into workbench libs Jan 25, 2018

jmthibault79 reviewed Jan 25, 2018

View reviewed changes

fix build script when pushing to develop

4d61565

jmthibault79 reviewed Jan 25, 2018

View reviewed changes

save git SHA during build script

905115f

jmthibault79 requested a review from rtitle January 25, 2018 17:18

jmthibault79 requested a review from aweng98 January 25, 2018 17:19

jmthibault79 mentioned this pull request Jan 25, 2018

use workbench libs #122

Closed

5 tasks

rtitle reviewed Jan 25, 2018

View reviewed changes

helgridly approved these changes Jan 25, 2018

View reviewed changes

rtitle reviewed Jan 25, 2018

View reviewed changes

rtitle approved these changes Jan 25, 2018

View reviewed changes

aweng98 approved these changes Jan 25, 2018

View reviewed changes

UrsaStutsman merged commit 09167fe into develop Jan 25, 2018

UrsaStutsman deleted the 2801-sam-redux branch January 25, 2018 20:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moving automation into workbench libs #134

Moving automation into workbench libs #134

UrsaStutsman commented Jan 24, 2018

jmthibault79 Jan 25, 2018

helgridly Jan 25, 2018

jmthibault79 Jan 25, 2018

jmthibault79 Jan 25, 2018

jmthibault79 Jan 25, 2018

helgridly Jan 25, 2018

jmthibault79 Jan 25, 2018

jmthibault79 Jan 25, 2018

jmthibault79 Jan 25, 2018

jmthibault79 commented Jan 25, 2018

jmthibault79 commented Jan 25, 2018

jmthibault79 commented Jan 25, 2018

rtitle Jan 25, 2018

rtitle Jan 25, 2018

rtitle Jan 25, 2018

UrsaStutsman Jan 25, 2018 •

edited

Loading

UrsaStutsman Jan 25, 2018

rtitle Jan 25, 2018

jmthibault79 Jan 25, 2018

rtitle Jan 25, 2018

rtitle left a comment

aweng98 left a comment

		@@ -1,6 +1,6 @@
		package org.broadinstitute.dsde.firecloud.page

		import org.broadinstitute.dsde.workbench.util.WebBrowserUtil


		case class LeoAuthToken(value: String)

		case object LeoAuthToken extends AuthToken {

Moving automation into workbench libs #134

Moving automation into workbench libs #134

Conversation

UrsaStutsman commented Jan 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmthibault79 commented Jan 25, 2018

jmthibault79 commented Jan 25, 2018

jmthibault79 commented Jan 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

UrsaStutsman Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rtitle left a comment

Choose a reason for hiding this comment

aweng98 left a comment

Choose a reason for hiding this comment

UrsaStutsman Jan 25, 2018 •

edited

Loading