Build Pipeline: Appveyor PR build issue

[bb-naveenkumar]

I have been getting the below issue while creating a pull request. Can anyone suggest how to resolve this. Are there some configuration issue or something?

Attached log location:
https://ci.appveyor.com/project/appsvc/azure-webjobs-sdk-881mw/builds/32077091

#2398

Build started
git clone -q https://github.com/Azure/azure-webjobs-sdk.git C:\projects\azure-webjobs-sdk-881mw
git fetch -q origin +refs/pull/2471/merge:
git checkout -qf FETCH_HEAD
dotnet --version
2.2.402
dotnet build WebJobs.sln -v q
Microsoft (R) Build Engine version 16.2.32702+c4012a063 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:48.20
vstest.console /logger:Appveyor /TestAdapterPath:bin /TestCaseFilter:"SecretsRequired!=true" bin/Microsoft.Azure.WebJobs.Host.UnitTests.dll bin/Microsoft.Azure.WebJobs.Host.FunctionalTests.dll bin/Microsoft.Azure.WebJobs.ServiceBus.UnitTests.dll bin/Dashboard.UnitTests.dll
Microsoft (R) Test Execution Command Line Tool Version 15.9.1
Copyright (c) Microsoft Corporation. All rights reserved.
The path 'bin' specified in the 'TestAdapterPath' is invalid. Error: The custom test adapter search path provided was not found, provide a valid path and try again.
The test source file "C:\projects\azure-webjobs-sdk-881mw\bin/Microsoft.Azure.WebJobs.Host.UnitTests.dll" provided was not found.
The test source file "C:\projects\azure-webjobs-sdk-881mw\bin/Microsoft.Azure.WebJobs.Host.UnitTests.dll" provided was not found.
The test source file "C:\projects\azure-webjobs-sdk-881mw\bin/Microsoft.Azure.WebJobs.Host.UnitTests.dll" provided was not found.
The test source file "C:\projects\azure-webjobs-sdk-881mw\bin/Microsoft.Azure.WebJobs.Host.UnitTests.dll" provided was not found.
Usage: vstest.console.exe [Arguments] [Options] [[--] ...]]
Description: Runs tests from the specified files.
Arguments:

[bb-naveenkumar]

Hi @brettsam can you please suggest something for the above if there is something that could resolve the test issue, I have seen you recently pushed new changes on the azure sdk repo. Thanks in advance!

[bb-naveenkumar]

I have found that the appveyor needs some permissions for it to run, how can I run it for the PRs in this repository. Seems like my fork commits can be built using appveyor by adding it to a new project. However not the PR. Anyone has any suggestions?

[bb-naveenkumar]

Seems like few changes needs to be made in the pipeline, can you guys let me know if that could be done, since PR builds will only succeed if pipeline is corrected or contributors could access the project for appveyor, also correct me if I am wrong anywhere. Thanks in advance!

[brettsam]

This is by-design. We don't allow external contributors to run the CI without one of the team members first adding a "full-ci" tag to the PR. It really should be nicer than this -- sorry it's caused you so much trouble.

[bb-naveenkumar]

It's alright atleast we got to know that it's not something that I have been missing on my part. Thank you. :)

[Chris-Johnston]

Hi, I think that this pattern can be a bit confusing for new contributors. I was definitely confused when I saw that my build failed for a reason unrelated to the changes in my branch. If full CI is still not an option for every PR, could this be communicated to new contributors through either:

• A message in a pull request template
• A warning message in the appveyor build script, before the script fails
• A message in the CONTRIBUTING.md file

Also, just curious, could you please share the reason why the CI won't run fully on every PR? CLA, or because this runs a full suite of integration tests?

[brettsam]

Your points are good ones -- I think there's a couple things we can do here. I'm going to re-open this and assign it to the next Sprint. Some things we can do:

1. Create a better experience when a non-team-member submits a PR. Instead of failing CI, try to get our "secretless" CI running again.
2. Explain in contributing.md that a team member will need to mark your PR valid for a full-ci run after an initial review.

We don't allow CI to run without team member intervention because our CI build uses secrets for some of the tests as they run against real services in Azure. We want someone from the team to take a look before we allow the tests to kick off to make sure they're not being printed to the run output anywhere.

[fabiocav]

@yojagad can you create issues to track migrating the build over to our DevOps infra?