Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures on travis #16

Closed
joevandyk opened this issue Jan 14, 2014 · 4 comments
Closed

Test failures on travis #16

joevandyk opened this issue Jan 14, 2014 · 4 comments

Comments

@joevandyk
Copy link
Contributor

https://travis-ci.org/joevandyk/que/builds/16960818

run 37 on 1.9.3 failed

jruby was failing because uninitialized constant PG::SyntaxError in migrations.

@chanks chanks closed this as completed in 1e8b7c4 Jan 14, 2014
@joevandyk
Copy link
Contributor Author

not sure if you want to look at the failure on run 37 on 1.9.3, guess we could see if it pops up again.

@chanks
Copy link
Collaborator

chanks commented Jan 14, 2014

I'm at 150 test iterations on my machine right now, it hasn't popped up
again. And if I can't reproduce it locally, there's not a lot I can do.
Whatever it is, it's not very common, so I'm just going to release. I'm
sure it'll turn up again at some point, though.

On Tue, Jan 14, 2014 at 2:45 PM, Joe Van Dyk notifications@github.comwrote:

not sure if you want to look at the failure on run 37 on 1.9.3.


Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-32316070
.

@joevandyk
Copy link
Contributor Author

Sounds good. Should we bump up the iterations on travis? How many runs does it usually take to see an error?

@chanks
Copy link
Collaborator

chanks commented Jan 15, 2014

I don't know, it's a crapshoot. I've had some errors take a thousand
iterations to pop up and then been unable to recreate them afterwards, even
using the same rspec seed on another thousand iterations.

I'm of two minds about it. Part of me wants to believe that the code can be
perfectly reliable, and that setting the iterations high (a few hundred?)
and being vigorous about investigating every failure will eventually result
in perfection. And part of me thinks that that approach will just aggravate
and stymie people who want to contribute and get unreproducible errors on
their pull requests for their trouble. And that if we stick to single spec
runs on Travis (just to catch the major oversights) and only take the time
to iterate thoroughly before a release, the results will be just fine and
it'll be a lot easier on everyone involved.

On Tue, Jan 14, 2014 at 3:48 PM, Joe Van Dyk notifications@github.comwrote:

Sounds good. Should we bump up the iterations on travis? How many runs
does it usually take to see an error?


Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-32320835
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants