Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-Memory Integration Testing #1390

Open
jaredcnance opened this issue Oct 17, 2017 · 9 comments

Comments

Projects
None yet
6 participants
@jaredcnance
Copy link

commented Oct 17, 2017

The problem

I would like to be able to run integration tests using the Azure Function runtime.

Workaround

I can use a FunctionRunner like this to fire up the runtime and then validate the results after some specified timeout.

There are several problems:

  • If I attempt to debug the test, I won't be able to set breakpoints within the actual function code since they are not in the same process. I have to run the test from the command line and add some while(Debugger.IsAttached == false) {/*...*/} hack.
  • I am required to set some arbitrary timeout and hope the function completes execution during that time
  • I cannot run a single function, all functions within the compiled function project will be started when the test host starts
  • requires func.exe to be installed on the testing server in a hardcoded location

Desired Solution

It would be great if we had a solution similar to AspNetCore that might look something like:

// arrange
var testHost = new TestHost(relativePathToCsproj);

// act
await testHost.RunFunctionAsync("MyFunction", maxInvocations: 1);

// assert...

A few things I would like to see included:

  • specify which function should be executed (and probably the binding as well)
  • specify the max number of times the function should be invoked before closing the host
  • a reasonable debugging story
  • assembly/type load errors should reproduce in tests. This probably means the test project should be executed by the function runtime

If the tools to do this already exist (within this project or any other), please let me know.

@paulbatum paulbatum added this to the Active Questions milestone Oct 18, 2017

@paulbatum

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

I would start by looking at our own e2e tests. For example we have this test fixture that should get you started.

If you end up building a test harness along the lines of what you outlined above that could be pretty useful in general, you should let us know.

Reopen this if you're still blocked.

@paulbatum paulbatum closed this Oct 18, 2017

@jaredcnance

This comment has been minimized.

Copy link
Author

commented Oct 18, 2017

It looks like that fixture is dependent upon System.Web.Http, will that be changing with the movement to dotnet core?

@paulbatum

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

Good question. I'm not sure what test fixture updates are planned, @fabiocav should be able to answer.

@jaredcnance

This comment has been minimized.

Copy link
Author

commented Oct 25, 2017

I don't want to try and craft a solution based on volatile infrastructure, but this is something that is sorely needed, especially when the risk of runtime errors is so high because of Azure/azure-functions-host#992

It looks like JobHost may be the thing I'm looking for in the future?

But then there is also this FunctionsLocalServer which seems to just start a new AspNetCore web host by spawning a new process to run the host via StartHostAction.

Can anyone provide me with the current design/direction so I don't do work that boxes me into a corner in future versions? Between this repository and the azure-functions-cli repo, I'm thoroughly confused as to how the runtime actually works.

@jaredcnance

This comment has been minimized.

Copy link
Author

commented Dec 1, 2017

Can we re-open this issue? I'm not sure why it is closed. I don't think a proper solution has been provided nor do I have enough information to produce one.

@paulbatum paulbatum reopened this Dec 7, 2017

@charisk

This comment has been minimized.

Copy link

commented Jan 30, 2018

I'm also facing the same challenge and agree with @jaredcnance - it would be great if there was some guidance here.

@jeffhollan

This comment has been minimized.

Copy link
Member

commented Jun 11, 2018

Spent some time this weekend playing with this during some travel. I shared my project here. As @paulbatum suggested it follows the same patterns used by the azure functions host project. It's a proof of concept but does allow me to execute the Azure Functions Host inside the test suite and not as an external executable. Has some kinks (wasn't able to ever get the breakpoints inside the function to fire when debugging) but should enable you to verify assembly loading as part of a test

https://github.com/jeffhollan/functions-test-helper

@s-krawczyk

This comment has been minimized.

Copy link

commented Apr 30, 2019

+1

@danhamlin

This comment has been minimized.

Copy link

commented Jun 4, 2019

Curious if @fabiocav has anything he is able to share about a proper test fixture for function apps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.