New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Function App - private assembly gets cached - updating file has no effect #953

Closed
marklauter opened this Issue Nov 19, 2016 · 11 comments

Comments

Projects
None yet
6 participants
@marklauter

marklauter commented Nov 19, 2016

Uploading private assembly to Azure Function /bin works, but subsequent updates don't take effect until the I kill the service process in Kudo. Restarting the app service in the Azure portal has no effect - service process must be killed. The symptom is that related changes in the Azure function will build after the assembly has been updated, such as accessing a new property in the assembly from the function, however the function will fail during runtime.

Repro steps

  1. create new function app
  2. create new event hubs trigger function
  3. create an assembly with a public class with at least one property
    3.a upload the assembly to function_name/bin and setup the reference in the function
    3.b the string passed to the function will be json from event hubs
    3.c use newtonsoft to deserialize the object
    3.d output the value of the property to the log
  4. create a console app that references the assembly from step 3
    4.a console app - instantiate the class and set the property with a value that will be logged by the event hub function
    4.b console app - write the object to event hubs
  5. the function will trigger, newtonsoft will decode the object, and the log will output the property value
  6. change the name of the property, update the assembly version, rebuild the assembly, upload the new version to the function_name/bin folder
  7. update the function with the new property name
  8. the function will compile
  9. run the console app
  10. the function will trigger and will error with something like this:
    2016-11-18T22:27:09.720 Exception while executing function: Functions.EventHubTriggerCSharp1. mscorlib: Exception has been thrown by the target of an invocation. EventHubTriggerCSharp1: Method not found: 'System.String AzureFunctionTest.DatagramMetadata.get_NewProperty()'.
    10.a try restarting the app service
    10.b no effect
  11. open kudo, kill all the processes
  12. wait a while for everything to restart
  13. run the console app
  14. the function triggers and runs as expected - logs output of renamed property

same behavior for .config files, except that changes to .config files never take effect, even after killing the service processes.

Expected behavior

I would expect any update to any file in the bin folder to be reflected in the behavior of the function.

Actual behavior

See repro steps

Known workarounds

Kill app service processes from Kudo.

Related information

Much more info and some sample code here:
http://stackoverflow.com/questions/40687108/azure-function-app-private-assembly-gets-cached-updating-file-has-no-effect

  • Programming language used
    C#
  • Links to source
    See StackOverflow link about
@fabiocav

This comment has been minimized.

Show comment
Hide comment
@fabiocav

fabiocav Nov 19, 2016

Member

Hey @marklauter, thank you for following up with the requested details here.

As mentioned on SO, I'll try to repro this as versioning the assembly should behave as expected and use the new version.

One question and one point of clarification:

  1. When you say you've restarted the site, which exact steps did you follow?
  2. Custom .config files are not supported in Azure Functions, so that is expected behavior.
Member

fabiocav commented Nov 19, 2016

Hey @marklauter, thank you for following up with the requested details here.

As mentioned on SO, I'll try to repro this as versioning the assembly should behave as expected and use the new version.

One question and one point of clarification:

  1. When you say you've restarted the site, which exact steps did you follow?
  2. Custom .config files are not supported in Azure Functions, so that is expected behavior.

@fabiocav fabiocav self-assigned this Nov 19, 2016

@marklauter

This comment has been minimized.

Show comment
Hide comment
@marklauter

marklauter Nov 20, 2016

Hi @fabiocav
From the function editor, I go to the Function App Settings tab. There I click the "Go to App Service Settings" button and the resulting blade has Stop and Restart buttons at the top.

marklauter commented Nov 20, 2016

Hi @fabiocav
From the function editor, I go to the Function App Settings tab. There I click the "Go to App Service Settings" button and the resulting blade has Stop and Restart buttons at the top.

@fabiocav

This comment has been minimized.

Show comment
Hide comment
@fabiocav

fabiocav Dec 9, 2016

Member

@marklauter I have been able to repro this and have identified the issue that causes this behavior and will have this addressed ASAP.

Thank you for your patience and for reporting this!

Member

fabiocav commented Dec 9, 2016

@marklauter I have been able to repro this and have identified the issue that causes this behavior and will have this addressed ASAP.

Thank you for your patience and for reporting this!

@marklauter

This comment has been minimized.

Show comment
Hide comment
@marklauter

marklauter Dec 9, 2016

@fabiocav That's awesome! Thank you for helping me out. I'm excited to start pushing Azure Functions on my clients - looks like a great way to save money on VMs.

marklauter commented Dec 9, 2016

@fabiocav That's awesome! Thank you for helping me out. I'm excited to start pushing Azure Functions on my clients - looks like a great way to save money on VMs.

fabiocav added a commit to fabiocav/azure-functions-host that referenced this issue Dec 9, 2016

fabiocav added a commit to fabiocav/azure-functions-host that referenced this issue Dec 9, 2016

fabiocav added a commit to fabiocav/azure-functions-host that referenced this issue Dec 9, 2016

fabiocav added a commit to fabiocav/azure-functions-host that referenced this issue Dec 13, 2016

@fabiocav fabiocav closed this in #1041 Dec 13, 2016

@fabiocav fabiocav removed the in progress label Dec 13, 2016

@chrisaut

This comment has been minimized.

Show comment
Hide comment
@chrisaut

chrisaut Jan 26, 2017

Has this rolled out yet? I feel like I'm seeing the same problem.

As a workaround, how do you kill the app in Kudu? I go to process explorer, and then command line and try taskkill pid or kill pid but I always get permission denied.

How can I force a reload of the app?

chrisaut commented Jan 26, 2017

Has this rolled out yet? I feel like I'm seeing the same problem.

As a workaround, how do you kill the app in Kudu? I go to process explorer, and then command line and try taskkill pid or kill pid but I always get permission denied.

How can I force a reload of the app?

@chrisaut

This comment has been minimized.

Show comment
Hide comment
@chrisaut

chrisaut Jan 27, 2017

Going to Function App Settings/Configure App Settings and changing a value there did force a reload. After that the correct dll seemed to get loaded.

So can you please confirm that the fix for this is not yet rolled out, or otherwise reopen this issue.

chrisaut commented Jan 27, 2017

Going to Function App Settings/Configure App Settings and changing a value there did force a reload. After that the correct dll seemed to get loaded.

So can you please confirm that the fix for this is not yet rolled out, or otherwise reopen this issue.

@brimmy11

This comment has been minimized.

Show comment
Hide comment
@brimmy11

brimmy11 Feb 27, 2017

@chrisaut I am seeing the same behaviour, changing/creating any app setting causes reload otherwise my DLL are not reloading.

brimmy11 commented Feb 27, 2017

@chrisaut I am seeing the same behaviour, changing/creating any app setting causes reload otherwise my DLL are not reloading.

@xmark

This comment has been minimized.

Show comment
Hide comment
@xmark

xmark Mar 20, 2017

I am also having this issue, is there anyway to clear the DLL cache?

xmark commented Mar 20, 2017

I am also having this issue, is there anyway to clear the DLL cache?

@brimmy11

This comment has been minimized.

Show comment
Hide comment
@brimmy11

brimmy11 Mar 20, 2017

@xmark I have been manually creating and then deleting an app setting to get it to refresh - I noticed in a recent Azure Function blog post they mentioning something about fixing a file locking issue which could very well be what is happening here.. https://blogs.msdn.microsoft.com/appserviceteam/2017/03/16/publishing-a-net-class-library-as-a-function-app/

brimmy11 commented Mar 20, 2017

@xmark I have been manually creating and then deleting an app setting to get it to refresh - I noticed in a recent Azure Function blog post they mentioning something about fixing a file locking issue which could very well be what is happening here.. https://blogs.msdn.microsoft.com/appserviceteam/2017/03/16/publishing-a-net-class-library-as-a-function-app/

@xmark

This comment has been minimized.

Show comment
Hide comment
@xmark

xmark Mar 20, 2017

@brimmy11 thanks mate, I'll try that and report back

xmark commented Mar 20, 2017

@brimmy11 thanks mate, I'll try that and report back

@xmark

This comment has been minimized.

Show comment
Hide comment
@xmark

xmark Mar 20, 2017

Ok, so that didnt work for me. But I did find the issue. I have 3 different functions that all reference the same DLL (a common set of domain objects and functions). I updated ONE of the DLLs in the bin directory for one of the functions only. (i.e. the other 2 functions still had the old version of the DLL, but I didnt think that it would make a different since I wasnt running them). I got the runtime error, but then just for a test I updated all the DLLs to the same version and it worked.

xmark commented Mar 20, 2017

Ok, so that didnt work for me. But I did find the issue. I have 3 different functions that all reference the same DLL (a common set of domain objects and functions). I updated ONE of the DLLs in the bin directory for one of the functions only. (i.e. the other 2 functions still had the old version of the DLL, but I didnt think that it would make a different since I wasnt running them). I got the runtime error, but then just for a test I updated all the DLLs to the same version and it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment