.NET Test invoking an Azure Function: An error occurred trying to start process 'func' with working directory. The system cannot find the file specified.

[jaliyaudagedara]

Description: (I am not sure whether this is the correct repo to log this issue, but I couldn't find a more suitable repo. If you think this doesn't belong here, appreciate it if you could point me to a more suitable repo.)

I have a .NET Test project, and inside the Test project, I am invoking an Azure Function as follows.

var appInfo = new ProcessStartInfo("func", $"start --port {port} --verbose")
{
	UseShellExecute = false,
	CreateNoWindow = true,
	RedirectStandardOutput = true,
	WorkingDirectory = projectDirectory.FullName,
	EnvironmentVariables =
	{
		["IsRunningIntegrationTests"] = "true"
	}
};
var app = new Process { StartInfo = appInfo };
app.Start();

I have installed Azure Functions Core Tools in the workflow before running the dotnet test step. From the workflow when I do dotnet test, which executes the above code, it fails with the following error.

[xUnit.net 00:00:00.[24](https://github.com/jaliyaudagedara/azure-functions-integration-tests/actions/runs/8001033163/job/21851461865#step:10:25)]     HelloAzureFunctions.Tests.Integration.HttpStartTests.HttpStart_Returns_CorrectResult [FAIL]
  Failed HelloAzureFunctions.Tests.Integration.HttpStartTests.HttpStart_Returns_CorrectResult [1 ms]
  Error Message:
   System.AggregateException : One or more errors occurred. (An error occurred trying to start process 'func' with working directory 'D:\a\azure-functions-integration-tests\azure-functions-integration-tests\tests\HelloAzureFunctions.Tests.Integration\bin\Debug\net8.0'. The system cannot find the file specified.) (The following constructor parameters did not have matching fixture data: AzureFunctionFixture azureFunctionFixture)
---- System.ComponentModel.Win32Exception : An error occurred trying to start process 'func' with working directory 'D:\a\azure-functions-integration-tests\azure-functions-integration-tests\tests\HelloAzureFunctions.Tests.Integration\bin\Debug\net8.0'. The system cannot find the file specified.
---- The following constructor parameters did not have matching fixture data: AzureFunctionFixture azureFunctionFixture
  Stack Trace:
  
----- Inner Stack Trace #1 (System.ComponentModel.Win32Exception) -----
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at HelloAzureFunctions.Tests.Integration.Fixtures.AzureFunctionFixture.StartApplication(Int32 port, DirectoryInfo projectDirectory) in D:\a\azure-functions-integration-tests\azure-functions-integration-tests\tests\HelloAzureFunctions.Tests.Integration\Fixtures\AzureFunctionFixture.cs:line 84
   at HelloAzureFunctions.Tests.Integration.Fixtures.AzureFunctionFixture..ctor() in D:\a\azure-functions-integration-tests\azure-functions-integration-tests\tests\HelloAzureFunctions.Tests.Integration\Fixtures\AzureFunctionFixture.cs:line 28
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

From the workflow, when I add a step to execute func, it's getting executed. So technically it's installed in the host (runner) where the workflow is running.

But it's failing to execute from the application code.

My workflow looks like below:

name: Run Integration Tests

on:
  push:
    branches: ["main"]

env:
  DOTNET_VERSION: '8.0.x'

jobs:
  build-and-test:
    runs-on: windows-latest

    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v3

    - name: Setup Node
      uses: actions/setup-node@v4   # Setting up node anyway to make sure npm install is good

    - name: Setup .NET ${{ env.DOTNET_VERSION }} Environment
      uses: actions/setup-dotnet@v3
      with:
        dotnet-version: ${{ env.DOTNET_VERSION }}

    - name: Build
      run: dotnet build

    - name: Install Azure Functions Core Tools
      run: |
        npm install -g azure-functions-core-tools@4 --unsafe-perm true

     # some steps
     
    - name: Func
      working-directory: ./tests/HelloAzureFunctions.Tests.Integration/bin/Debug/net8.0
      run: func    # this works

    - name: Dir
      working-directory: ./tests/HelloAzureFunctions.Tests.Integration/bin/Debug/net8.0
      run: dir    # this displays that the directory contains the Azure Function executables

    - name: Test
      working-directory: ./tests/HelloAzureFunctions.Tests.Integration/bin/Debug/net8.0
      continue-on-error: true
      run: dotnet test ./HelloAzureFunctions.Tests.Integration.dll

Task version:
N/A

Platform:

• Windows

Runner type:

• Hosted

Current runner version: '[2](https://github.com/jaliyaudagedara/azure-functions-integration-tests/actions/runs/8002139291/job/21854736706#step:1:2).[3](https://github.com/jaliyaudagedara/azure-functions-integration-tests/actions/runs/8002139291/job/21854736706#step:1:3)13.0'
Operating System
  Microsoft Windows Server 2022
  10.0.203[4](https://github.com/jaliyaudagedara/azure-functions-integration-tests/actions/runs/8002139291/job/21854736706#step:1:4)[8](https://github.com/jaliyaudagedara/azure-functions-integration-tests/actions/runs/8002139291/job/21854736706#step:1:9)
  Datacenter
Runner Image
  Image: windows-2022
  Version: 20240218.2.0
  Included Software: https://github.com/actions/runner-images/blob/win22/20240218.2/images/windows/Windows2022-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20240218.2
Runner Image Provisioner
  2.0.341.1

Repro steps:
Repo: jaliyaudagedara/azure-functions-integration-tests

Expected behavior:
I have the same concept working on Azure DevOps pipelines on windows-latest, can't understand why the application code can't pick up globally installed azure-functions-core-tools in GitHub workflows.

[jaliyaudagedara]

Tried func --version and func.exe --version, didn't work.

var appInfo = new ProcessStartInfo("func", $"--version") { ... }
var appInfo = new ProcessStartInfo("func.exe", $"--version") { ... }

Tried npm --version, didn't work.

var appInfo = new ProcessStartInfo("npm", $"--version") { ... }

Then tried this powershell.exe echo 'Hello World', and it worked. Interesting!

var appInfo = new ProcessStartInfo("powershell.exe", $"echo 'Hello World'") { ... }

[jaliyaudagedara]

Started a GitHub community discussion: https://github.com/orgs/community/discussions/109546

[aparnajyothi-y]

Hello @jaliyaudagedara, Thank you for creating this issue and we will look into it :)

[jaliyaudagedara]

I was able to get around the issue by doing the following in test code,

var appInfo = new ProcessStartInfo("powershell.exe", $"func start --port {port} --verbose") { ... }

And the dotnet test was successfully executed.

integration-test-output.log

Still curious, why directly supplying the command doesn't work in GitHub workflows, but it works in Azure DevOps pipelines.

[shaanmugapriya]

Hello @jaliyaudagedara,

Thank you once again for creating the issue.

Upon investigating further, When you install a tool in a command-line environment, like the Azure Functions Core Tools using npm install -g, the tool's executable files are usually added to a directory on your system.

For your command-line environment to find these executables when you type a command, the directory containing the executables needs to be included in an environment variable called PATH. The PATH variable is a list of directories that your command-line environment checks when you enter a command.

Now, the way this PATH variable is handled can differ between environments:

GitHub Actions: When you install a tool in one step of a GitHub Actions workflow (like npm install -g azure-functions-core-tools@4), the tool's executables are not automatically added to the PATH for the subsequent steps. This means, in a later step when your .NET code tries to start a new process with the func command, it can't find the func executable and throws an error.

Azure DevOps Pipelines: In contrast, it appears that Azure DevOps Pipelines automatically add the directory of the installed tool to the PATH. So, when you install the Azure Functions Core Tools, the func command is immediately available for subsequent tasks in the pipeline.

This is why changing your .NET code to start a powershell.exe process (which is always available in the PATH) and passing the func command as an argument works in GitHub Actions. The powershell.exe process can find and run the func command because it's executing it within its own environment where the PATH has been set correctly.

I hope this helps clarify the difference. The handling of the PATH environment variable is a low-level detail of these CI/CD environments and is usually abstracted away from users, but it can cause issues like the one you're seeing.

Please feel free to reach us if any further clarification required. Thanks!!!

[jaliyaudagedara]

@shaanmugapriya,

Thank you very much for the detailed explanation.

That explains everything. Thanks again, very much appreciated!

Closing the issue as it's the expected behavior per the design.