Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix JENKINS-41134 #63

Merged
merged 3 commits into from Jan 27, 2017

Conversation

Projects
None yet
3 participants
@slide
Copy link
Member

slide commented Jan 25, 2017

This adds a test for JENKINS-41134, fix which will make the test pass will come after.

slide added some commits Jan 25, 2017

@slide slide changed the title Add test for JENKINS-41134 Fix JENKINS-41134 Jan 25, 2017

@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="TestSuite_second" tests="1" errors="0" failures="1" skip="0" timestamp="2012-04-05T10:50:00">

This comment has been minimized.

Copy link
@jtnord

jtnord Jan 25, 2017

Member

So the interesting thing with the Jenkins results is that the test suite/ classname and test name are identical (duplicate between the passing and failing test) - so allthough this is a good test, it's not reproducing the issue as observed

This comment has been minimized.

Copy link
@slide

slide Jan 25, 2017

Author Member

True, I will add another test. I did verify on an actual build of Jenkins with the Jenkinsfile that the fix does resolve that issue, but a completely valid test is a good idea.

This comment has been minimized.

Copy link
@slide

slide Jan 25, 2017

Author Member

Are we worried about the case when the same exact test runs and has the same exact timestamp? If the two tests with the same name have a different timestamp, they are considered a different instance of the test. If they have the same name, and the same timestamp they are considered the same. What is the likelihood that the same test suite will run at the same exact time on two nodes?

This comment has been minimized.

Copy link
@jtnord

jtnord Jan 26, 2017

Member

my gut says that the probability is low (even though the the timestamp has only second granularity).
At least for the reported failure (as provisioning the windows node takes a while).

Now for something running tests in a parallel branch on the same node (e.g. with different browsers or some flag) then the probability is probably higher - but I think that can be handled as a different issue - as it may break use cases where a user runs unit tests, and then integration (maven failsafe) on the same workspace and runs the junit step in between...

This comment has been minimized.

Copy link
@jamesdh

jamesdh Apr 27, 2017

This exact scenario you described above appears to have bitten us. We have mocha tests that run extraordinarily fast, and sure enough we have three different test suites with the same name, same timestamp, but one of them has failures. Jenkins was only occasionally reporting the failures. Guessing it was just the luck of the draw as to which one it picked to represent the pass/fail state.

Is there an issue created to represent this scenario?

@slide

This comment has been minimized.

Copy link
Member Author

slide commented Jan 27, 2017

Any other feedback? If not, I will merge this today

@jtnord

jtnord approved these changes Jan 27, 2017

@jtnord

This comment has been minimized.

Copy link
Member

jtnord commented Jan 27, 2017

Thanks for looking into this Alex.

@slide slide merged commit 631c413 into jenkinsci:master Jan 27, 2017

1 check passed

Jenkins This pull request looks good
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.