Fix problems running E2E tests locally #957
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/azp run |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As called out in #930, we have recently begun to have local E2E test failures. Upon investigation, the problem was determined that the test logs are being written to the folder we are using as the
BuildDropPath, and that causes the SBOM validation to fail. This seems to be a recent change (bug?) in Visual Studio--we have been able to specify the--results-directoryparameter when running from the command line, but attempts to change it when running from inside Visual Studio have proven unsuccessful.This PR works around the problem by copying the existing folder to a temporary folder before running the tests, then cleaning up the temporary folder after the tests complete. Since the
Microsoft.Sbom.Targets.Testsproject has bothnet472andnet80targets, we need to ensure that each target has its own directory.The
AbstractGenerateSbomTaskInputTests.csandAbstractGenerateSbomTaskTests.cstests were refactored to allow the names of the concrete classes to be passed into the abstract base classes. The old code usedCurrentDirectory, which no longer seemed correct, so I renamed it toTestBuildDropPath. The SPDX 3.0 E2E tests will probably use these same abstract classes.I'm not thrilled with having 3 copies of the
Xcopyfunction. At one point I had it down to 2 copies (one per project), but once it was written, the extra overhead was worse than having the extra copy. Happy to entertain options here if people have strong feelings.