Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restructure jenkins ext tests for origin/build speed up #21504

Merged
merged 1 commit into from Nov 28, 2018

Conversation

gabemontero
Copy link
Contributor

a restructure of the jenkins pipeline tests given @smarterclayton 's complaints on the time spent

Rationale

  • the feature/builds ginkgo focus string remain in the version used for openshift origin's openshift/build
  • all the jenkins and jenkins plugin e2e tests focus on strings like openshift pipeline build which remain in both files
  • the tests run in the openshift/build suite are focused on regression testing of sync plugin / fabric client and openshift api server compatability
  • the client plugin tests can be off loaded from openshift/build since presumably the oc binary to api server compatibility is vetted elsewhere ... problems in the groovy shim should be properly covered by jenkins e2e test, which will run the same set of tests as before
  • anything which runs the sample pipeline templates that leverage maven, mongo, etc. are offloaded to the jenkins e2e
  • I vetted tests that remained in openshift/build that they actually only do things like mvn --version and sh exit 0
  • more complex flows centered more on plugin code paths vs. api server compatibility got moved out of openshift/build
  • some of the delete/prune tests had subtle differences, so I split the difference there

/assign @bparees

@openshift-ci-robot openshift-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 15, 2018
@gabemontero
Copy link
Contributor Author

/test extended_builds

@gabemontero
Copy link
Contributor Author

/refresh

@gabemontero
Copy link
Contributor Author

ok how the hell do i get the extended builds job scheduled ...grepping on openshift/release the command appears to be /test extended_builds

@bparees
Copy link
Contributor

bparees commented Nov 15, 2018

/test e2e-gcp-builds

but it's already running. extended-builds isn't a thing anymore.

@gabemontero
Copy link
Contributor Author

e2e-aws failure: more IP pain: aws_eip.nat_eip.4: Error creating EIP: AddressLimitExceeded: The maximum number of addresses has been reached.

@gabemontero
Copy link
Contributor Author

/retest e2e-aws

@bparees
Copy link
Contributor

bparees commented Nov 15, 2018

@gabemontero as i'm unlikely to have time to give this a solid review, please have @adambkaplan give it a second set of eyes, but basically i trust you, if you think this is giving us the coverage we need and it's speeding up (or parallelizing) the execution, i'm fine with it.

/approve

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 15, 2018
@bparees
Copy link
Contributor

bparees commented Nov 15, 2018

/assign @adambkaplan

@gabemontero
Copy link
Contributor Author

sounds good @bparees

@gabemontero
Copy link
Contributor Author

OK e2e-gcp-build was successful and illustrated the reduced set of jenkins tests

e2e-gcp had a smattering of failures, a few of which were in the build space, including jenkins, but it appears they were interrupted .... overall runs timing out ? .... I was unable to say for sure

retesting to see about consistency

@gabemontero
Copy link
Contributor Author

/retest

@bparees
Copy link
Contributor

bparees commented Nov 19, 2018

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 19, 2018
@bparees
Copy link
Contributor

bparees commented Nov 19, 2018

@smarterclayton fyi, see if this helps w/ the build test time concerns you had after it merges.

@gabemontero
Copy link
Contributor Author

gabemontero commented Nov 19, 2018

@smarterclayton - while the removal of [Feature:Builds] has excluded some of the tests from e2e-gcp-builds as expected, those tests are still showing up in e2e-gcp.

Is there some special focus string we can apply to the tests that leverage all the actual samples like mongodb, etc. so they don't get run in the conformance test?

The intent of this PR is to run those in the openshift/jenkins e2e ...

We only want to run our optimized test samples (that circumvent all these slow samples) in e2e-gcp and e2e-gcp-builds

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Nov 19, 2018
@gabemontero
Copy link
Contributor Author

@bparees and @smarterclayton got us sorted out in forum-devex ... PR updated to exclude jenkins tests from conformance e2e-gcp

@bparees
Copy link
Contributor

bparees commented Nov 19, 2018

@gabemontero once you confirm the test isn't running in gcp let me know and i'll lgtm it

@gabemontero
Copy link
Contributor Author

/test integration

@gabemontero
Copy link
Contributor Author

e2e-gcp run https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/21504/pull-ci-openshift-origin-master-e2e-gcp/1965 completed successfully @bparees and the jenkins tests are correctly omitted

@bparees
Copy link
Contributor

bparees commented Nov 19, 2018 via email

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 19, 2018
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

2 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

for several days now the e2e-gcp still is correct in that none of the jenkins tests are in it

the failures fall across several components in that time ... in the build space, they are generally registry / docker related:

	Copying blob sha256:e11c8e053fb5d1761f80fbfe7369765ecf2942fb2d975d3dd1d2792fcd8cabbd
		error: build error: Error reading blob sha256:e11c8e053fb5...lid status code returned when fetching blob 404 (Not Found)
W1125 11:26:16.615429     874 dockerimagelookup.go:236] Docker registry lookup failed: unknown blob
error:  local file access failed with: stat docker.io/openshift/hello-openshift: no such file or directory

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

9 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gabemontero
Copy link
Contributor Author

Clayton uncovered a dockerhub spat of flakiness that mos likely explains the e2e-gcp build failures I noted in #21504 (comment)

They have appeared to addressed that and it is no longer occurring.

Just a couple of networking items in the last run.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

12 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 756f7b4 into openshift:master Nov 28, 2018
@gabemontero gabemontero deleted the jenkins-ext-tst branch November 28, 2018 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants