extended tests for jenkins openshift V3 plugin #6329

gabemontero · 2015-12-15T20:51:15Z

bparees · 2015-12-15T21:02:33Z

test/extended/jenkins/jenkins_plugin.go

+	return data
+}
+
+var _ = g.Describe("jenkins: plugin: run job leverging openshift pipeline plugin", func() {


gabemontero · 2015-12-15T23:08:51Z

@bparees updates based on 1st set of comments

bparees · 2015-12-15T23:44:23Z

test/extended/util/framework.go

@@ -189,22 +189,22 @@ var CheckImageStreamTagNotFoundFn = func(i *imageapi.ImageStream) bool {
 func WaitForADeployment(client kclient.ReplicationControllerInterface,


I think i would have done this differently.... either:

just check the start time and calculate and end time, and then if the end time has passed at the top of the outer loop

get rid of the the Watch entirely and just wrap the list() in a wait.Poll check (which is basically what WaitForABuild does).

with the existing logic it's pretty hard to follow how long it's really going to end up waiting, based on expiration of watches. also 3 minutes between polls seems like a lot.

went with option 1) ... will push in a sec

mfojtik · 2015-12-16T15:24:23Z

test/extended/util/image_helpers.go

+//NewApp performs an oc new-app invocation from the supplied json file
+func NewApp(jsonFilePath string, oc *CLI) error {
+	err := oc.Run("new-app").Args(jsonFilePath).Execute()
+	return err


just return the oc.Run here :-)

is this worth to have special function? I think reading the code will tell you what this does.... having NewApp() makes you go find the definition if something explode :)

gabemontero · 2015-12-16T21:18:54Z

hack/install-assets.sh failed ... never got to the extended tests in the extended test run

bparees · 2015-12-16T22:58:08Z

test/extended/util/framework.go

@@ -189,7 +189,9 @@ var CheckImageStreamTagNotFoundFn = func(i *imageapi.ImageStream) bool {
 func WaitForADeployment(client kclient.ReplicationControllerInterface,
 	name string,
 	isOK, isFailed func(*kapi.ReplicationController) bool) error {
-	for {
+	startTime := time.Now()
+	endTime := startTime.Add(3 * time.Minute)


bump this up to something like 15 or 20 minutes. we don't want to introduce flakes if we have existing extended tests that take a long time waiting for a deployment for some reason.

i'm also surprised 3 minutes is long enough for you test, given that it basically has to get through the entire imagebuild+deploy step in 3 minutes to pass.

(3 minutes or the watch timeout, whichever is longer.)

Will do - it worked for me locally but i did not measure how long it
typically took

On Wednesday, December 16, 2015, Ben Parees notifications@github.com
wrote:

In test/extended/util/framework.go
#6329 (comment):

@@ -189,7 +189,9 @@ var CheckImageStreamTagNotFoundFn = func(i _imageapi.ImageStream) bool {
func WaitForADeployment(client kclient.ReplicationControllerInterface,
name string,
isOK, isFailed func(_kapi.ReplicationController) bool) error {

for {

startTime := time.Now()

endTime := startTime.Add(3 * time.Minute)

bump this up to something like 15 or 20 minutes. we don't want to
introduce flakes if we have existing extended tests that take a long time
waiting for a deployment for some reason.

i'm also surprised 3 minutes is long enough for you test, given that it
basically has to get through the entire imagebuild+deploy step in 3 minutes
to pass.

(3 minutes or the watch timeout, whichever is longer.)

—
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/pull/6329/files#r47846861.

I'm guessing the watch doesn't get closed because there's not much etcd
change log activity, so you don't come back to the top of the loop, so the
3 minute limit doesn't come into play in your particular scenario.

Ben Parees | OpenShift
On Dec 16, 2015 10:20 PM, "Gabe Montero" notifications@github.com wrote:

In test/extended/util/framework.go
#6329 (comment):

@@ -189,7 +189,9 @@ var CheckImageStreamTagNotFoundFn = func(i _imageapi.ImageStream) bool {
func WaitForADeployment(client kclient.ReplicationControllerInterface,
name string,
isOK, isFailed func(_kapi.ReplicationController) bool) error {

for {

startTime := time.Now()

endTime := startTime.Add(3 * time.Minute)

Will do - it worked for me locally but i did not measure how long it
typically took
… <#151adf41a51c75cf_>
On Wednesday, December 16, 2015, Ben Parees notifications@github.com
wrote: In test/extended/util/framework.go <#6329 (comment)
https://github.com/openshift/origin/pull/6329#discussion_r47846861>: >
@@ -189,7 +189,9 @@ var CheckImageStreamTagNotFoundFn = func(i
_imageapi.ImageStream) bool { > func WaitForADeployment(client
kclient.ReplicationControllerInterface, > name string, > isOK, isFailed
func(_kapi.ReplicationController) bool) error { > - for { > + startTime :=
time.Now() > + endTime := startTime.Add(3 * time.Minute) bump this up to
something like 15 or 20 minutes. we don't want to introduce flakes if we
have existing extended tests that take a long time waiting for a deployment
for some reason. i'm also surprised 3 minutes is long enough for you test,
given that it basically has to get through the entire imagebuild+deploy
step in 3 minutes to pass. (3 minutes or the watch timeout, whichever is
longer.) — Reply to this email directly or view it on GitHub <
https://github.com/openshift/origin/pull/6329/files#r47846861>.

—
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/pull/6329/files#r47865373.

gabemontero · 2015-12-17T13:46:06Z

the test only extended keep failing in a early, just after build-go.sh completes successfully, in a weird way, way before it gets to the extended test.

tried refreshing the branch; if still fails, going to remove the existing test flags and just try a generic test job.

gabemontero · 2015-12-17T14:19:20Z

ok ... it looks like it got to try and run the extended test, but it seems to fail immediately with an error that is not obvious to me at first blush:

Running TEST_REPORT_DIR=/tmp/openshift-extended-tests/junit/extended test/extended/core.sh --ginkgo.focus="jenkins"...
package k8s.io/kubernetes/pkg/util
imports github.com/davecgh/go-spew/spew
imports github.com/docker/libcontainer/cgroups/fs
imports github.com/docker/docker/pkg/mount
imports github.com/docker/docker/pkg/units
imports github.com/docker/docker/pkg/units
imports github.com/docker/docker/pkg/units: cannot find package "github.com/docker/docker/pkg/units" in any of:
/usr/lib/golang/src/github.com/docker/docker/pkg/units (from $GOROOT)
/data/src/github.com/docker/docker/pkg/units (from $GOPATH)

I'm going to see about clearing all the prior test flags, see if I can get full fledged test going.

mfojtik · 2015-12-17T14:20:11Z

@gabemontero why do you import "github.com/docker/docker/pkg/units" ?

mfojtik · 2015-12-17T14:21:42Z

OK this seems like kubernetes imports that package... that is weird, do you have godeps setup properly?

gabemontero · 2015-12-17T14:23:15Z

@mfojtik I don't import "github.com/docker/docker/pkg/units" .... as far as i know i do, I did not make any godeps changes as part of this PR ... in any event, wouldn't the test job on jenkins be the one that pulls the various godeps?

mfojtik · 2015-12-17T14:30:58Z

@gabemontero I just ran that command and it worked for me, that means that you might have something wrong with your environment... can you check the value of GOPATH?

bparees · 2015-12-17T16:49:20Z

lgtm, waiting for extended test framework to be fixed up.

gabemontero · 2016-01-07T16:43:37Z

[testonlyextended][extended:core(jenkins)]

gabemontero · 2016-01-07T17:37:53Z

back to testing, having pulled in pr #6558 while it is waiting to merge, and uncovered a timing issue panic in the wait for jenkins to come up logic ... pushing an update for retest shortly.

gabemontero · 2016-01-07T18:31:36Z

ok ... the new extended test just passed !! ... Going to push the back out of the 6558 fix, confirm things are still good

gabemontero · 2016-01-07T19:43:04Z

OK that's two straight successful extended test runs. And the travis hiccup appears to be some sort of go 1.4 flake (it works in 1.5).

@bparees IMO - this is ready to hit the merge button on - PTAL at your convenience / discretion

bparees · 2016-01-07T21:14:17Z

test/extended/jenkins/jenkins_plugin.go

+	}
+}
+
+// as g.It's are added conceivably will be with new jenkins job def's


scrambled godoc

or scrambled comment. not sure what it was intended to be.

just deleted - no longer pertinent as I recall

bparees · 2016-01-07T21:20:49Z

one nit and a question about the deployment that's being verified. otherwise lgtm.

gabemontero · 2016-01-08T02:04:19Z

ok ... i've pushed the updates in response to the one nit and question

bparees · 2016-01-08T04:10:53Z

lgtm [merge]

openshift-bot · 2016-01-08T04:15:23Z

[Test]ing while waiting on the merge queue

gabemontero · 2016-01-08T15:37:07Z

the https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin/8311/ failure came from hack/../test/cmd/volumes.sh, prior to the launching of the extended tests

the https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_origin/4555/ failure came from hack/../test/cmd/images.sh

can we re-hit the merge button or force the merge of this new extended test (which passed here: https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin/8309/) ?

bparees · 2016-01-08T16:03:34Z

i can keep hitting it, but until we sort out the underlying etcd flake issue, don't hold your breath
[merge]

(I don't want to force merge even though this would be low risk, it's just a bad pattern to get into. we need to fix the merge queue)

gabemontero · 2016-01-08T16:08:25Z

@bparees roger that

bparees · 2016-01-08T22:38:21Z

[merge]

deads2k · 2016-01-09T01:25:13Z

[test]

openshift-bot · 2016-01-09T10:05:20Z

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_origin/4584/) (Image: devenv-rhel7_3110)

deads2k · 2016-01-09T21:22:33Z

Gofmt, not a flake

openshift-bot · 2016-01-09T22:45:14Z

Evaluated for origin testonlyextended up to 2b35c51

openshift-bot · 2016-01-09T22:45:26Z

Evaluated for origin test up to 2b35c51

gabemontero · 2016-01-09T22:45:50Z

yep, found 1 more thing to fix last night, failed to gofmt ... just pushed

if needed somebody please hit merge button when get the chance - thx

bparees · 2016-01-09T22:51:19Z

[merge]

openshift-bot · 2016-01-09T22:55:16Z

Evaluated for origin merge up to 2b35c51

openshift-bot · 2016-01-09T23:20:53Z

continuous-integration/openshift-jenkins/testonlyextended SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin/8413/) (Extended Tests: core(jenkins))

Merged by openshift-bot

openshift-bot · 2016-01-09T23:56:27Z

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin/8414/) (Extended Tests: core(jenkins))

bparees reviewed Dec 15, 2015
View reviewed changes

gabemontero force-pushed the jenkinsExtendedTests branch from 6020f06 to c0f132f Compare December 15, 2015 23:08

bparees reviewed Dec 15, 2015
View reviewed changes

gabemontero force-pushed the jenkinsExtendedTests branch from c0f132f to 5170ed4 Compare December 16, 2015 02:54

mfojtik reviewed Dec 16, 2015
View reviewed changes

gabemontero force-pushed the jenkinsExtendedTests branch from 5170ed4 to a1f4791 Compare December 16, 2015 15:48

bparees reviewed Dec 16, 2015
View reviewed changes

gabemontero force-pushed the jenkinsExtendedTests branch 2 times, most recently from 2b5af4b to 870339a Compare December 17, 2015 13:30

gabemontero force-pushed the jenkinsExtendedTests branch from 870339a to 4b2f077 Compare January 7, 2016 16:35

gabemontero force-pushed the jenkinsExtendedTests branch from 4b2f077 to 5b9ee55 Compare January 7, 2016 16:48

gabemontero force-pushed the jenkinsExtendedTests branch from 5b9ee55 to e592ac4 Compare January 7, 2016 17:49

gabemontero force-pushed the jenkinsExtendedTests branch from e592ac4 to d23a24f Compare January 7, 2016 18:31

bparees reviewed Jan 7, 2016
View reviewed changes

gabemontero force-pushed the jenkinsExtendedTests branch from d23a24f to 9f1765b Compare January 8, 2016 02:03

gabemontero force-pushed the jenkinsExtendedTests branch from 9f1765b to 8cb273b Compare January 8, 2016 22:15

extended tests for jenkins openshift V3 plugin

2b35c51

gabemontero force-pushed the jenkinsExtendedTests branch from 8cb273b to 2b35c51 Compare January 9, 2016 22:43

openshift-bot pushed a commit that referenced this pull request Jan 9, 2016

Merge pull request #6329 from gabemontero/jenkinsExtendedTests

ad80e1f

Merged by openshift-bot

openshift-bot merged commit ad80e1f into openshift:master Jan 9, 2016

gabemontero deleted the jenkinsExtendedTests branch January 10, 2016 22:41

gabemontero mentioned this pull request Feb 16, 2016

Jenkins plugin e2e hangs and runs forever #7277

Closed

		@@ -189,22 +189,22 @@ var CheckImageStreamTagNotFoundFn = func(i *imageapi.ImageStream) bool {
		func WaitForADeployment(client kclient.ReplicationControllerInterface,

extended tests for jenkins openshift V3 plugin #6329

extended tests for jenkins openshift V3 plugin #6329

Conversation

gabemontero commented Dec 15, 2015

Choose a reason for hiding this comment

gabemontero commented Dec 15, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabemontero commented Dec 16, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabemontero commented Dec 17, 2015

gabemontero commented Dec 17, 2015

mfojtik commented Dec 17, 2015

mfojtik commented Dec 17, 2015

gabemontero commented Dec 17, 2015

mfojtik commented Dec 17, 2015

bparees commented Dec 17, 2015

gabemontero commented Jan 7, 2016

gabemontero commented Jan 7, 2016

gabemontero commented Jan 7, 2016

gabemontero commented Jan 7, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparees commented Jan 7, 2016

gabemontero commented Jan 8, 2016

bparees commented Jan 8, 2016

openshift-bot commented Jan 8, 2016

gabemontero commented Jan 8, 2016

bparees commented Jan 8, 2016

gabemontero commented Jan 8, 2016

bparees commented Jan 8, 2016

deads2k commented Jan 9, 2016

openshift-bot commented Jan 9, 2016

deads2k commented Jan 9, 2016

openshift-bot commented Jan 9, 2016

openshift-bot commented Jan 9, 2016

gabemontero commented Jan 9, 2016

bparees commented Jan 9, 2016

openshift-bot commented Jan 9, 2016

openshift-bot commented Jan 9, 2016

openshift-bot commented Jan 9, 2016