Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added logging of output for flaky build tests #6240

Merged
merged 1 commit into from
Dec 9, 2015
Merged

added logging of output for flaky build tests #6240

merged 1 commit into from
Dec 9, 2015

Conversation

PI-Victor
Copy link
Contributor

related to Issue: #6215
also an older conversation about this #5912 (comment)
@bparees @mfojtik PTAL

for i := 0; i <= 3; i++ {
out, err = oc.Run("start-build").Args(buildName, "--follow", "--wait").Output()
if err != nil {
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be good to log the error here so we know it is retrying it

@liggitt
Copy link
Contributor

liggitt commented Dec 9, 2015

I don't think I'm a fan of this... do we know the root cause? Are we expecting customers to add retry logic around builds to get reliable results as well?

@PI-Victor PI-Victor changed the title added retry logic for flaky build tests added logging of output for flaky build tests Dec 9, 2015
@PI-Victor
Copy link
Contributor Author

we added a way to log the output of the build whenever it fails so that we can identify what the problem is rather than add a retry logic to it.
The main reason why this fails right now, is because bundle fails.

Unable to open /etc/scl/conf/ror40!
F1205 08:39:53.504032       1 builder.go:70] Build error: The command '/bin/sh -c scl enable ror40 "bundle install"' returned a non-zero code: 1

Basically ginkgo still flushes to stdout on failure, but with this change we can make sure that we catch the problem easier next time and after that we can remove it.

@bparees
Copy link
Contributor

bparees commented Dec 9, 2015

lgtm. [merge]

@openshift-bot
Copy link
Contributor

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_origin/4276/) (Image: devenv-rhel7_2910)

@openshift-bot
Copy link
Contributor

Evaluated for origin merge up to 638175a

openshift-bot pushed a commit that referenced this pull request Dec 9, 2015
@openshift-bot openshift-bot merged commit a20ad02 into openshift:master Dec 9, 2015
@PI-Victor
Copy link
Contributor Author

@rhcarvalho @bparees @mfojtik see my above comment as to why the bundle install failed

@bparees
Copy link
Contributor

bparees commented Dec 10, 2015

@PI-Victor that failure was, i think, because we were running scl enable ror40 on the ruby22 image which doesn't have ror40. if we're still doing that anywhere, we should track it down and fix it. eg i fixed one instance here:
#6221

@bparees
Copy link
Contributor

bparees commented Dec 10, 2015

possibly that is the same issue you hit, depending when the build was from.

@PI-Victor
Copy link
Contributor Author

well the link to the actual build from jenkins you mentioned in the issue, is from 5th of Dec.
https://ci.openshift.redhat.com/jenkins/job/origin_extended/377/ so yeah, it hit the exact issue with the scl enable. should i revert the changes made in this PR, or just keep them for the time being?

@bparees
Copy link
Contributor

bparees commented Dec 10, 2015

@PI-Victor I don't think the change hurts anything, we're still going to want that info if we hit failures. so i'd leave it.

@PI-Victor PI-Victor deleted the ipalade/fix_flaky_tests branch May 3, 2016 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants