Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for bypass of not proceeding deployment check jenkins extended tests given update center sync delay is now on startup #19008

Conversation

gabemontero
Copy link
Contributor

@gabemontero gabemontero commented Mar 16, 2018

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 16, 2018
@bparees
Copy link
Contributor

bparees commented Mar 16, 2018

hm. this is going to make these tests always wait 3 minutes before they have any hope of progressing.

should we be changing the waitfordeploymentconfig logic? is it just flat out wrong that it currently aborts when progressing is false, based on your deployment api investigation?

I think this is worth a discussion with @mfojtik and team.

@bparees
Copy link
Contributor

bparees commented Mar 16, 2018

i'm going to lgtm this for now to unbreak the tests but i still want to followup on changing the waitfordeploymentconfig logic so our jenkins tests don't have to do this mandatory 3 minute delay.

/lgtm

/hold
(for passing tests)

@openshift-ci-robot openshift-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 16, 2018
@gabemontero
Copy link
Contributor Author

gabemontero commented Mar 16, 2018 via email

@gabemontero
Copy link
Contributor Author

Well we got not progressing errors even with this change:

[image_ecosystem][jenkins][Slow] openshift pipeline plugin jenkins-plugin test context jenkins-plugin test multitag DSL [Suite:openshift] 12m25s

go run hack/e2e.go -v -test --test_args='--ginkgo.focus=\[image\_ecosystem\]\[jenkins\]\[Slow\]\sopenshift\spipeline\splugin\s\sjenkins\-plugin\stest\scontext\s\s\sjenkins\-plugin\stest\smultitag\sDSL\s\[Suite\:openshift\]$'

/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:154
Expected error:
    <*errors.errorString | 0xc420b58590>: {s: "not progressing"}
    not progressing
not to have occurred
/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:206
				
				Click to see stdout/stderrfrom focus_parallel_03.xml

Filter through log files
[image_ecosystem][jenkins][Slow] openshift pipeline plugin jenkins-plugin test context jenkins-plugin test trigger build with slave [Suite:openshift] 12m23s

go run hack/e2e.go -v -test --test_args='--ginkgo.focus=\[image\_ecosystem\]\[jenkins\]\[Slow\]\sopenshift\spipeline\splugin\s\sjenkins\-plugin\stest\scontext\s\s\sjenkins\-plugin\stest\strigger\sbuild\swith\sslave\s\[Suite\:openshift\]$'

/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:154
Expected error:
    <*errors.errorString | 0xc4201b85d0>: {s: "not progressing"}
    not progressing
not to have occurred
/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:206
				
				Click to see stdout/stderrfrom focus_parallel_02.xml

Filter through log files

Will get more aggressive in allowing for adjustments of existing logic

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Mar 16, 2018
@gabemontero gabemontero changed the title relax deployment wait for jenkins extended tests given update center … allow for bypass of not proceeding deployment check jenkins extended tests given update center sync delay is now on startup Mar 16, 2018
@gabemontero
Copy link
Contributor Author

new version pushed that allows for not proceeding check to be optional

@bparees
Copy link
Contributor

bparees commented Mar 16, 2018

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 16, 2018
@gabemontero
Copy link
Contributor Author

The bump of the jenkins LTS seems to have caused another problem (at least that is the most likely explanation for this exception, since we have not changed the login plugin lately):

java.io.IOException: java.lang.RuntimeException: Failed to serialize jenkins.model.Jenkins#securityRealm for class hudson.model.Hudson
	at hudson.XmlFile.write(XmlFile.java:200)
	at jenkins.model.Jenkins.save(Jenkins.java:3168)
	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3174)
	at jenkins.model.Jenkins.setAuthorizationStrategy(Jenkins.java:2544)
	at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm.updateAuthorizationStrategy(OpenShiftOAuth2SecurityRealm.java:859)
	at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftPermissionFilter.doFilter(OpenShiftPermissionFilter.java:210)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:564)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
	at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
	at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to serialize jenkins.model.Jenkins#securityRealm for class hudson.model.Hudson
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:193)
	... 70 more
Caused by: java.lang.RuntimeException: Failed to serialize org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm#transport for class org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
	... 83 more
Caused by: java.lang.UnsupportedOperationException: Refusing to marshal com.google.api.client.http.javanet.NetHttpTransport for security reasons; see https://jenkins.io/redirect/class-filter/
	at hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:543)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
	... 92 more

Don't know yet why this did not show up in the image tests or my manual starting of the image at 2.101.

Will study a little be more, and there may be a change in the login plugin to avoid this, but the quick fix is probably to revert the lts bump, then try to sort this out.

@gabemontero
Copy link
Contributor Author

Marking this field transient might be the longer term solution:

Caused by: java.lang.RuntimeException: Failed to serialize org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm#transport for class org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm

@gabemontero
Copy link
Contributor Author

Duh ... bringing up the image outside a pod means the login plugin does not insert itself as the sec realm

@gabemontero
Copy link
Contributor Author

@gabemontero
Copy link
Contributor Author

/retest

…tests given update center sync delay is now on startup
@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Mar 17, 2018
@gabemontero
Copy link
Contributor Author

gabemontero commented Mar 17, 2018

last run looked better (no exceptions saving the oauth sec realm where we are back to 2.89.4 for now)

the image_eco/jenkins-plugin.go runs still seem more prone to the deploy to's ... but they still run at 512Mi vs. our build/pipeline.go runs. Bumped image_eco/jenkins-plugin.go to more mem, and added pod log dump on jenkins deploy failure

In parallel, i'm testing locally:
a) i believe i have a login plugin change that will allow moving back to jenkins v2.107
b) but i'm still seeing some deploy timeouts in my local runs - seems more constrained than the ci/aws runs... going to look into, besides running with PARALLEL_NODES=1 on my laptop
i) moving the image's groovy init onto a separate thread ... perhaps that allows jenkins startup to proceed faster
ii) if that doesn't do it, figure our right deployment setting, modify jenkins template some more to allow more deploy time / reach out to master team as needed
iii) or give up once again on incurring the updater center churn on startup vs. first login

@gabemontero
Copy link
Contributor Author

OK, look better wrt the update center / GC stuff.

The image eco failure was in a jenkins test, but the problem was unrelated to the JVM. There was a timeout adding the edit role to the service account:

/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:154
Expected error:
    <*util.ExitError | 0xc42195e150>: {
        Cmd: "oc policy --config=/tmp/extended-test-jenkins-plugin-t9f42-lnzpd-user.kubeconfig --namespace=extended-test-jenkins-plugin-t9f42-lnzpd add-role-to-user edit system:serviceaccount:extended-test-jenkins-plugin-t9f42-lnzpd-jenkins:jenkins",
        StdErr: "Error from server: timeout",
        ExitError: {
            ProcessState: {
                pid: 109903,
                status: 256,
                rusage: {
                    Utime: {Sec: 0, Usec: 310356},
                    Stime: {Sec: 0, Usec: 60405},
                    Maxrss: 62672,
                    Ixrss: 0,
                    Idrss: 0,
                    Isrss: 0,
                    Minflt: 14998,
                    Majflt: 0,
                    Nswap: 0,
                    Inblock: 0,
                    Oublock: 0,
                    Msgsnd: 0,
                    Msgrcv: 0,
                    Nsignals: 0,
                    Nvcsw: 985,
                    Nivcsw: 1012,
                },
            },
            Stderr: nil,
        },
    }
    exit status 1
not to have occurred
/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/image_ecosystem/jenkins_plugin.go:246

@gabemontero
Copy link
Contributor Author

the extended builds were 2 more instances of the service not being considered ready in 600 seconds

going to get another sample while I try the other alternatives noted in #19008 (comment)

@gabemontero
Copy link
Contributor Author

/retest

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 19, 2018
@gabemontero
Copy link
Contributor Author

ok, we actually got clean build/image_eco system runs this time, even without the upcoming jenkins image changes

given the latest build/image_ecosystem overnight runs both had a couple of intermittent not progressing errors, shall we merge this in the interim @bparees ?

I removed the debug stuff already

@bparees
Copy link
Contributor

bparees commented Mar 19, 2018

/hold cancel
/lgtm

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Mar 19, 2018
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bparees, gabemontero

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

7 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

fyi, none of the last few failures have been with the jenkins tests

@gabemontero
Copy link
Contributor Author

gabemontero commented Mar 20, 2018

the last conformance install failure was flake #17883

UPDATE - same test, but different problem from the above flake. The cause was different:

/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:28
Expected error:
    <*util.ExitError | 0xc42061db30>: {
        Cmd: "oc new-build --config=/tmp/extended-test-s2i-build-root-gb8n9-tmw7m-user.kubeconfig --namespace=extended-test-s2i-build-root-gb8n9-tmw7m -D FROM centos/nodejs-6-centos7\nUSER 0 --name nodejsroot",
        StdErr: "error: can't lookup images: Timeout: request did not complete within allowed duration\nerror: unable to locate any images in image streams, local docker images with name \"centos/nodejs-6-centos7\"\n\nThe 'oc new-build' command will match arguments to the following types:\n\n  1. Images tagged into image streams in the current project or the 'openshift' project\n     - if you don't specify a tag, we'll add ':latest'\n  2. Images in the Docker Hub, on remote registries, or on the local Docker engine\n  3. Git repository URLs or local paths that point to Git repositories\n\n--allow-missing-images can be used to force the use of an image that was not matched\n\nSee 'oc new-build -h' for examples.",
        ExitError: {
            ProcessState: {
                pid: 98557,
                status: 256,
                rusage: {
                    Utime: {Sec: 0, Usec: 265538},
                    Stime: {Sec: 0, Usec: 44610},
                    Maxrss: 71380,
                    Ixrss: 0,
                    Idrss: 0,
                    Isrss: 0,
                    Minflt: 15038,
                    Majflt: 0,
                    Nswap: 0,
                    Inblock: 0,
                    Oublock: 0,
                    Msgsnd: 0,
                    Msgrcv: 0,
                    Nsignals: 0,
                    Nvcsw: 1869,
                    Nivcsw: 304,
                },
            },
            Stderr: nil,
        },
    }
    exit status 1
not to have occurred
/tmp/openshift/build-rpms/rpm/BUILD/origin-3.10.0/_output/local/go/src/github.com/openshift/origin/test/extended/builds/s2i_root.go:35

@bparees @legionus @dmage - should i open a new issue to track this?

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

various Stage PROVISION CLOUD RESOURCES [00h 10m 24s] failed. failures

/retest

@gabemontero
Copy link
Contributor Author

more provision cloud failures ... will wait a bit, see if clouds heal

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

provision test cluster problem

/test gcp

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

ok tests pass again ... let's see if we get lucky and they do so again now that we are back on the merge queue

@openshift-merge-robot
Copy link
Contributor

/test all [submit-queue is verifying that this PR is safe to merge]

@openshift-merge-robot
Copy link
Contributor

Automatic merge from submit-queue (batch tested with PRs 19008, 18727, 19038).

@openshift-merge-robot openshift-merge-robot merged commit eda7e84 into openshift:master Mar 21, 2018
@gabemontero gabemontero deleted the relax-jenkins-deploy-verify branch March 21, 2018 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. sig/developer-experience size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants