Unable to Deploy

[jazzyray]

Unable to deploy. Repeated attempts. Stopped, stopped ASE, restated, re-deployed. All failures.
Please see trace below.

Versions:

Azure Functions Core Tools (2.0.1-beta.39)
Function Runtime Version: 2.0.12095.0
<azure.functions.maven.plugin.version>1.0.0-beta-6-SNAPSHOT</azure.functions.maven.plugin.version>
<azure.functions.java.library.version>1.0.0-beta-5</azure.functions.java.library.version>

Snippet:

2018-09-18T07:47:52 An error has occurred during web site deployment.
2018-09-18T07:47:52 Error: Failed to change file that is currently being used "D:\home\site\wwwroot\api_companyview-1.0-SNAPSHOT.jar"

D:\Program Files (x86)\SiteExtensions\Kudu\77.10830.3542\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"
2018-09-18T07:47:52 Error occurred, type: error, text: Error: Failed to change file that is currently being used "D:\home\site\wwwroot\api_companyview-1.0-SNAPSHOT.jar"

D:\Program Files (x86)\SiteExtensions\Kudu\77.10830.3542\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd", stackTrace: at Kudu.Core.Infrastructure.Executable.ExecuteInternal(ITracer tracer, Func2 onWriteOutput, Func2 onWriteError, Encoding encoding, String arguments, Object[] args) in C:\Kudu Files\Private\src\master\Kudu.Core\Infrastructure\Executable.cs:line 235

at Kudu.Core.Infrastructure.Executable.ExecuteWithProgressWriter(ILogger logger, ITracer tracer, String arguments, Object[] args) in C:\Kudu Files\Private\src\master\Kudu.Core\Infrastructure\Executable.cs:line 137

Investigative information

Please provide the following:

• Timestamp: 2018-09-18T07:45:27
• Function App name: emb-api-dev-20180917
• Function name(s) (as appropriate): many
• Invocation ID:
• Region: West Europre

Repro steps

Provide the steps required to reproduce the problem:

deploy using

mvn clean package azure-functions:deploy

Expected behavior

Provide a description of the expected behavior.

Deployment and function operating correctly.

Actual behavior

Provide a description of the actual behavior observed.

Unable to deploy or start the function.

Attached trace file
logtrace3.txt

[jazzyray]

Clean attempt. Stop everything, start. deploy --> logfile
logtrace4.txt

[jazzyray]

I changed the deploy type to msdeploy. The deploy seems to be working now. (See trace below)
However, there lots of errors in the logs. It certainly does not look like a clean deploy.

There is an issue with zip deploy. imo

@jdneo Should you perhaps change the default deploy type back to msdeploy? It looks shaky to me.

Cheers
Jem

Trace from msdeploy -->
logtrace5.txt

[jdneo]

@jazzyray MSDeploy is not the recommended deploy method to deploy the function artifact.

Could you provide repro steps to us which can help us diagnostic?

[jazzyray]

Also @jdneo Function app settings are not passed correctly using msdeploy.
These work fine with zip deploy.

<appSettings>
                        <property>
                            <name>FUNCTIONS_EXTENSION_VERSION</name>
                            <value>${azure.functions.java.runtime.version}</value>
                        </property>
                        <property>
                            <name>FUNCTIONS_WORKER_RUNTIME</name>
                            <value>java</value>
                        </property>
                        <property>
                            <name>ES_USERNAME</name>
                            <value>${env.ES_USERNAME}</value>
                        </property>
                        <property>
                            <name>ES_PASSWORD</name>
                            <value>${env.ES_PASSWORD}</value>
                        </property>
                        <property>
                            <name>GDB_USERNAME</name>
                            <value>${env.GDB_USERNAME}</value>
                        </property>
                        <property>
                            <name>GDB_PASSWORD</name>
                            <value>${env.GDB_PASSWORD}</value>
                        </property>
                    </appSettings>

[jazzyray]

@jdneo You can see the failed zip deploy in the first comment. Looks like something wrong in the function host. I do not want to use msdeploy however I have been forced to as zip is not working.

What information do want to be able to re-create the issue? Can you examine the logs to start with.
Notice:

2018-09-18T07:47:52 An error has occurred during web site deployment.
2018-09-18T07:47:52 Error: Failed to change file that is currently being used "D:\home\site\wwwroot\api_companyview-1.0-SNAPSHOT.jar"

D:\Program Files (x86)\SiteExtensions\Kudu\77.10830.3542\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"
2018-09-18T07:47:52 Error occurred, type: error, text: Error: Failed to change file that is currently being used "D:\home\site\wwwroot\api_companyview-1.0-SNAPSHOT.jar"

D:\Program Files (x86)\SiteExtensions\Kudu\77.10830.3542\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd", stackTrace: at Kudu.Core.Infrastructure.Executable.ExecuteInternal(ITracer tracer, Func2 onWriteOutput, Func2 onWriteError, Encoding encoding, String arguments, Object[] args) in C:\Kudu Files\Private\src\master\Kudu.Core\Infrastructure\Executable.cs:line 235

at Kudu.Core.Infrastructure.Executable.ExecuteWithProgressWriter(ILogger logger, ITracer tracer, String arguments, Object[] args) in C:\Kudu Files\Private\src\master\Kudu.Core\Infrastructure\Executable.cs:line 137

I suspect its not a maven plugin issue.
Regards

[jazzyray]

I guess you also have the zip artifacts on the function host if you want to re-create?

[jdneo]

We will first try to repro this issue and to see what's happening. @Flanker32 will take a look at this issue.

[Flanker32]

@jazzyray
Hi, could you please share me your project to reproduce this issue? i tried functions created by plugin and it seems work well.
Besides, it seems that you are using SNAPSHOT version of the function, we have release the function 1.0.0-beta-6 and you may use the release version.

[jazzyray]

@Flanker32 I have deployed using the beta-6 release (NON Snapshot) using zip.
The same issue occurs. (see trace).
Swapping back to msdeploy fixes it.

On sharing the project. The code is not public.

Perhaps the difference between what you are testing and my project is that I deploy multiple zips to the same function host. Where each zip contains a sub-set of the functions. Have you tested this? Perhaps this is the issue.

Perhaps I can screen share with you, rather than sharing the code, my client would be reluctant to share the code.

Regards
Jem
Attached trace file
logtrace2.txt

[jdneo]

@jazzyray Thank you for the feedback. I think the logs are detailed enough. (For now, no need to paste the logs for the later discussion, it's hard to read the context. 😃 )

Some more questions might help us understand/repro this issue.

• What's your app service plan for the function app. (consumption or dedicated.)
  You can find it here
  

• Could you repro this issue if the deployed file is a smaller one, for example, only contains one function.

[jazzyray]

@jdneo

On this: "What's your app service plan for the function app. (consumption or dedicated.)
You can find it here"

Subscription ID: 9649282e-6ae7-4488-b646-195a841fca77
Dedicated Custom ASE (App Service Plan): ACEMWPUASPEMB01 (Premium: 1 WP1)

We need a custom ASE for VNet/Subnet etc access. (We cannot use VNet/VPN connections).

On this:
"Could you repro this issue if the deployed file is a smaller one, for example, only contains one function."

A Single zip deploys to a single Function host appears to be OK.

It appears the issue is related to when you deploy multiple zips to the same function host. (Locking etc). E.g. Single ASE, Single function Host, two zips deployed to the Function host, where each zip contains one or many functions:

ASE(A) -> Function Host(B) --> zip(C) -- function (D,E)
ASE(A) -->Function Host(B) --> zip(F) -- function (G,H)

Cheers
Jem

[jdneo]

@pragnagopa We might need your help about this issue.

The customer is encounting the problem that he could not deploy zip artifact. According to the logs from the service side, you can see a lot of:

2018-09-20T09:03:55 Error: Failed to change file that is currently being used "D:\home\site\wwwroot\XXXXX.jar"

Should we stop the function app everytime before we do zipdeploy in order to release the jar file?

[pragnagopa]

cc: @davidebbo

[davidebbo]

The support for App_Offline was added around 9/21, so I suggest testing if this still repros.

Regardless, switching to run from package is recommended here. All they have to do it set WEBSITE_RUN_FROM_PACKAGE to 1, and then continue to use zipdeploy as they do today.

[pragnagopa]

@jazzyray -Please retry after updating pom.xml to include app setting for run from package

<appSettings>
  <property>
     <name>FUNCTIONS_EXTENSION_VERSION</name>
     <value>~2</value>
   </property>
   <property>
     <name>WEBSITE_RUN_FROM_PACKAGE</name>
     <value>1</value>
   </property>
</appSettings>