Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuring travis/buildbot testing to be more robust #1384

Closed
6 of 8 tasks
durack1 opened this issue Jun 12, 2015 · 19 comments
Closed
6 of 8 tasks

Configuring travis/buildbot testing to be more robust #1384

durack1 opened this issue Jun 12, 2015 · 19 comments
Assignees
Milestone

Comments

@durack1
Copy link
Member

@durack1 durack1 commented Jun 12, 2015

@jbeezley @doutriaux1 @aashish24 this creates a dedicated issue to attempt to resolve a bunch of errors that continue to recur in the travis and buildbot build tests as discussed in #1381 and a number of other issues/PRs.
Specific changes to implement:

  • Add redhat 6.6 platform (6.6 - crunchy.llnl.gov - ALL) to the buildbot suite
  • Add redhat 6.7 platform (6.7 - oceanonly.llnl.gov - NOGUI) to the buildbot suite
  • Add mac 10.10 platform (10.10 - macbot.llnl.gov - LEAN) to the buildbot suite
  • Add mac 10.11 platform (10.11 - ALL) to the buildbot suite
  • Add redhat 7.x platform (7.1 - pcmdi10.llnl.gov - ALL available) to the buildbot suite (#457)
  • Use travis_wait before the cmake call to prevent failures due to long waits between build steps (see here and here) - Is there a switch for cmake which will allow retrying dropped source download attempts too?
  • Deal with timeouts during cdms tests to opendap servers (see here, here and here)
  • Deal with race conditions in vtk image checks (see here, here, here and here - these errors show where the right-side image pixels are randomized on the canvas also here, an attempt to add a time.sleep(1) to prevent a race condition in another test was used in #1341 here) - fixed by #1393

This might also be useful:
http://docs.travis-ci.com/user/build-timeouts/

@chaosphere2112
Copy link
Contributor

@chaosphere2112 chaosphere2112 commented Jun 15, 2015

@jbeezley It looks like all of the failures are on the same buildbot; the test-laptop one. Is that a particularly slow device?

@chaosphere2112
Copy link
Contributor

@chaosphere2112 chaosphere2112 commented Jun 15, 2015

Also, just submitted #1393 to help with some race conditions in my tests.

@durack1
Copy link
Member Author

@durack1 durack1 commented Jun 22, 2015

@potter2, @doutriaux1 and I were wondering if you may have any under-utilized mac hardware that would be suitable as a mac-buildbot candidate? We currently don't have any mac hardware available to run build tests, and a 10.9, 10.10 and possibly even a 10.11 machine would certainly be helpful if available..

@potter2
Copy link

@potter2 potter2 commented Jun 23, 2015

I really don't have anything. Sorry

Jerry

Sent from my iPad

On Jun 22, 2015, at 11:25 AM, Paul J. Durack <notifications@github.commailto:notifications@github.com> wrote:

@potter2https://github.com/potter2, @doutriaux1https://github.com/doutriaux1 and I were wondering if you may have any under-utilized mac hardware that would be suitable as a mac-buildbot candidate? We currently don't have any mac hardware available to run build tests, and a 10.9, 10.10 and possibly even a 10.11 machine would certainly be helpful if available..


Reply to this email directly or view it on GitHubhttps://github.com//issues/1384#issuecomment-114208344.

@durack1
Copy link
Member Author

@durack1 durack1 commented Sep 1, 2015

@potter2 this is what @doutriaux1 and I spoke to you about last week, getting another mac to run ALL build tests.. @doutriaux1 already has a server configured, but we need more clients and you indicated you have a mac machine at home that could be used within the buildbot test suite..?

@potter2
Copy link

@potter2 potter2 commented Sep 1, 2015

Paul,
I am at GSFC/NASA this week but I will be home next week. Maybe we can get it set up then. Will the build work on a home network?

Jerry

Gerald (Jerry) Potter
NASA Goddard Space Flight Center
E-mail gerald.potter@nasa.govmailto:gerald.potter@nasa.gov
Office 28:S241
Phone 707.315.8314
Analysis and Data Consultant
Department GSFC:606.2

On Sep 1, 2015, at 11:49 AM, Paul J. Durack <notifications@github.commailto:notifications@github.com> wrote:

@potter2https://github.com/potter2 this is what @doutriaux1https://github.com/doutriaux1 and I spoke to you about last week, getting another mac to run build tests.. @doutriaux1https://github.com/doutriaux1 already has a server configured, but we need more clients and you indicated you have a mac machine at home that could be used within the buildbot test suite..?


Reply to this email directly or view it on GitHubhttps://github.com//issues/1384#issuecomment-136767023.

@durack1
Copy link
Member Author

@durack1 durack1 commented Sep 1, 2015

Great @potter2 yeah the build will work, as long as it is permanently connected.. @doutriaux1 can explain specifics, but simply when a pull request is issued through github, that new branch is built on the buildbot suite, in which we currently don't have a mac ALL test running.. If you have a fairly powerful machine that is always on, we could then use this to test an ALL mac build and improve the testing and build robustness on mac platforms..

@potter2
Copy link

@potter2 potter2 commented Sep 1, 2015

I need to think about the mac connection. I’ll get back t you next week..

Jerry

Gerald (Jerry) Potter
NASA Goddard Space Flight Center
E-mail gerald.potter@nasa.govmailto:gerald.potter@nasa.gov
Office 28:S241
Phone 707.315.8314
Analysis and Data Consultant
Department GSFC:606.2

On Sep 1, 2015, at 1:44 PM, Paul J. Durack <notifications@github.commailto:notifications@github.com> wrote:

Great @potter2https://github.com/potter2 yeah the build will work, as long as it is permanently connected.. @doutriaux1https://github.com/doutriaux1 can explain specifics, but simply when a pull request is issued through github, that new branch is built on the buildbot suite, in which we currently don't have a mac ALL test running.. If you have a fairly powerful machine that is always on, we could then use this to test an ALL mac build and improve the testing and build robustness on mac platforms..


Reply to this email directly or view it on GitHubhttps://github.com//issues/1384#issuecomment-136809349.

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 1, 2015

@durack1 no it is not a given, for example at my place ATT blocks me from talking over 9982... I tried to open the port but no luck so far, even tried using other ports. From talking to ATT they have no idea my port stays blocked even after I tell my router to open it... So it's not 100% sure.

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 1, 2015

@potter2 any progress with ucdavis?

@durack1
Copy link
Member Author

@durack1 durack1 commented Sep 1, 2015

@doutriaux1 could a routing rule be setup?

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 1, 2015

I tried that and talked with ATT they were baffled it didn't work...

@jbeezley
Copy link

@jbeezley jbeezley commented Sep 1, 2015

There is always ngrok.

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 1, 2015

oh that's looks nice, maybe i will try that.

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 2, 2015

will add a mac 10.11 when it's out officially (and bug have been ironed out)

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 2, 2015

i think we aare pretty much abandoning travis at that point @chaosphere2112 are we getting a pro licence or not? If not let's dump travis

@durack1
Copy link
Member Author

@durack1 durack1 commented Sep 17, 2015

@doutriaux1 Xcode 7 was released today which goes along with OS X 10.11, so probably worth dropping that into the buildbot suite as soon as an OS X machine becomes available..

@durack1 durack1 added this to the 2.4 milestone Sep 17, 2015
@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 17, 2015

Once the lab has an approved image for the green network will do.

@doutriaux1
Copy link
Contributor

@doutriaux1 doutriaux1 commented Sep 18, 2015

@doutriaux1 doutriaux1 closed this Sep 18, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants