Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid AppVeyor build failures due to problem uploading JUnit XML #19243

Merged
merged 1 commit into from May 1, 2019

Conversation

Projects
None yet
2 participants
@jasonrudolph
Copy link
Member

commented Apr 30, 2019

Closes #18994

Description of the Change

As seen in #18994, our AppVeyor builds periodically fail due to problems uploading the JUnit XML output. Since we already have JUnit XML output for all of our builds on Azure DevOps, and since we want to retire AppVeyor and move to Azure DevOps, it seems pragmatic to resolve this issue by removing the JUnit XML processing from the AppVeyor build. This pull request currently takes this approach to resolving #18994.

Alternate Designs

We could enhance the uploading logic to gracefully ignore errors that occur during upload and/or retry the upload when it fails. However, since we're hoping to migrate off of AppVeyor, this doesn't feel like a particularly productive endeavor.

Possible Drawbacks

When a test fails on AppVeyor, developers will need to look for the test failure in the build log (screenshot) instead of finding it on the "Tests" tab (screenshot).

Verification Process

  • Verify that Azure DevOps has offers the JUnit XML goodies for all platforms -- #19243 (comment)
  • Ensure that the AppVeyor build still passes and that you can still see all the build output in the AppVeyor log
@jasonrudolph

This comment has been minimized.

Copy link
Member Author

commented Apr 30, 2019

  • Verify that Azure DevOps has offers the JUnit XML goodies for all platforms

To verify this, I pushed up a temporary branch with some platform-specific tests that would intentionally fail:

if (process.platform === 'linux') {
it('runs linux-specific tests', () => {
fail()
})
} else if (process.platform === 'darwin') {
it('runs darwin-specific tests', () => {
fail()
})
} else {
it('runs windows-specific tests', () => {
fail()
})
}

I then ran those tests on Azure DevOps, and verified that the "Tests" tab shows each of these tests:

Screen Shot 2019-04-30 at 4 17 12 PM

In other words, any test that was failing on Windows on AppVeyor should also be visible in the Windows output on Azure DevOps, so I don't think we're losing any meaningful functionality by removing the JUnit XML support on AppVeyor.

@jasonrudolph jasonrudolph marked this pull request as ready for review Apr 30, 2019

@jasonrudolph jasonrudolph requested a review from smashwilson Apr 30, 2019

@jasonrudolph

This comment has been minimized.

Copy link
Member Author

commented Apr 30, 2019

@smashwilson: Given your work in this area in #15124, would you mind reviewing this change?

@smashwilson
Copy link
Member

left a comment

👍 Sure, seems reasonable.

Redundancy with Azure isn't everything -- we have seen tests that fail here and not on Azure and vice versa, for whatever reason. (Isn't CI fun?) But I think we can deal with a bit of scrolling until we can move fully off of AppVeyor.

@jasonrudolph jasonrudolph merged commit 4d7f7de into master May 1, 2019

2 checks passed

Atom Pull Requests #20190430.6 succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details

@jasonrudolph jasonrudolph deleted the no-junit-xml-on-appveyor branch May 1, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.