Azure WebApp Publish failed due to env not found error

[YoungjaeKim]

Symptom

I tried to publish Flask app to Azure WebApp, but failed due to the error message below. Sadly, default Azure Flask template also failed to publish.

A virtual environment is required to deploy dependencies to Microsoft Azure Web Sites. Either add a virtual environment or add true to your project if you have no dependencies other than the Python standard library.

Reproduce Steps

1. git pull https://github.com/azureappserviceoss/FlaskAzure.git (azure default template)
2. In Visual Studio, New > Project > From Existing Python Code > select pulled folder.
3. In setup wizard, set as Flask app and Detect virtual environment.
4. It seems failed to detenct env directory. Project has Python27 (global default) as environment.
5. Right-click Python Environment and click Add Existing Virtual Environment.
6. Select env directory and it shows "We could not find a base interpreter at D:\Python27" error.
7. When right-click Python Environment and click Add/Remove Python Environments, the dialog shows env (unavailable) in environment list.

Questions

1. Why does VisualStudio point D:\Python27 in Step 6 and how can I fix it? I do not have D: drive and cannot find hard-coded line for that.
2. Where can I put <SuppressWebSitesVirtualEnvError>true</SuppressWebSitesVirtualEnvError> in web.config? I cannot find any documentation and example.

[huguesv]

Virtual environments have hard coded paths to the base python path, which can cause problems if you try to reuse them across machines like you are doing here.

I would recommend that you delete it and recreate it using PTVS. Since there's a requirements.txt at the root, PTVS will check the "install packages" in the new virtual env dialog by default.

FYI SuppressWebSitesVirtualEnvError element is meant for the .pyproj file, not the web.config file.

[zooba]

Also so you're aware, we're probably going to remove the virtualenv publishing functionality in the next version (with Visual Studio "15"). We'll recommend the process outlined at https://aka.ms/PythonOnAppService for getting started (and hopefully streamline it a bit by then, but even now we know customers are happier with that approach).

[YoungjaeKim]

Hi @zooba
I spent hours to setup right web.config with reference to your link but I failed to run up due to the following error.

scriptProcessor could not be found in application configuration

Could you guide me the correct web.config setup?

I uploaded my repository (PTVS created) as GitHub Public repo.

Reproduce Steps

1. Create new WebApp Basic tier + Install Site Extension Python x86 3.5.2 + turn on Diagnostic Logs.
2. Setup GitHub deployment with my repository which created from PTVS Flask project in VS2015 Enterprise: https://github.com/YoungjaeKim/AzureFlaskDemo
3. run d:\home\python35\python.exe -m pip install --upgrade -r d:\home\site\wwwroot\requirements.txt in Kudu CMD
4. <handler> error occurred.

You can see my web.config in here; https://github.com/YoungjaeKim/AzureFlaskDemo/blob/master/web.config

I successfully run it with Http Platform handler guide but it has another problem. It shows almost 8 seconds to process the GET / request. I think it is by http://stackoverflow.com/q/11150343/361100 problem but AFAIK WebApp does not allow custom hosts.

[zooba]

I think you need to remove the rewrite rules from your web.config. Otherwise all your URLs are going to have a handler.fcgi prefix that won't be handled properly.

However, the error makes it look like the applicationHost.xdt transform in the site extension isn't running properly. There should be some log files in a LogFiles\Transform folder that might have some extra information, but I've honestly not seen this fail before.

For the HTTP platform handler, do the second and subsequent requests also take 5-8 seconds? It wouldn't surprise me if there was a slow initial startup time but things are fine once the server is running.

[YoungjaeKim]

@zooba Thanks for supporting me.

HTTP platform handler

I tested HTTP platform handler again and found that it requires initial warm up longer than I expected. I supposed the warm-up would be enough only at the first request but almost 10 times gives fast response.

Transform log

There's no special message about transformation but let me attach the content.

2016-11-03T07:18:52 sandboxproc.exe C:\DWASFiles\Sites\youngjaeflasktest\Temp\applicationhost.config
2016-11-03T07:18:52 env XPROC_TYPENAME=Microsoft.Web.Hosting.Transformers.ApplicationHost.SiteExtensionHelper, Microsoft.Web.Hosting, Version=7.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
2016-11-03T07:18:52 env XPROC_METHODNAME=Transform
2016-11-03T07:18:52 Start 'python352x86' site extension transform
2016-11-03T07:18:52 StartSection Executing InsertIfMissing (transform line 12, 9)
2016-11-03T07:18:52 on /configuration/system.webServer/fastCgi/application
2016-11-03T07:18:52 Applying to 'fastCgi' element (no source line info)
2016-11-03T07:18:52 EndSection Done executing InsertIfMissing
2016-11-03T07:18:53 Successful 'D:\home\SiteExtensions\python352x86\applicationHost.xdt' site extension transform
2016-11-03T07:18:53 sandboxproc.exe complete successfully. Ellapsed = 309.00 ms

FastCGI revisited

By following your comment, I removed <rewrite> block in web.config but still got the same error as below.

scriptProcessor could not be found in application configuration

I'd like to make run Flask with FastCGI as it has more integrated pipeline than self server. Could you test with my repository? or check my web.config again?

Thanks

[zooba]

There's nothing wrong with your web.config - it would seem that something strange is going on with the site itself. If it's easy for you to do, deleting and redeploying the whole thing may fix the issue, otherwise let's dig deeper.

Can you check with the Kudu console that running D:\home\Python35\python.exe -V displays the version number (3.5.2), and that D:\home\Python35\python.exe -c "import wfastcgi" doesn't fail? It's possible that the site extension has failed to deploy properly for some reason, which might lead to the error you're seeing. If python.exe isn't actually there, the only way I'm aware of to clean up the site extension is to manually delete it and reinstall. You'll have to delete D:\home\Python35 if it exists, and also D:\home\SiteExtensions\python352x86, then go through the process of adding it again.

There may be some other log files (in a Kudu directory, I believe) that indicate why it didn't deploy properly. If you wanted to dig through those and see if there's anything interesting, that will help us make sure it doesn't happen in the future.

[YoungjaeKim]

Hi. @zooba I got a fresh news. please read.

Test Steps

1. Create vanilla WebApp in Basic tier (East Asia region)
2. Turn on Diagnostics Logs to File system
3. Add Extension Python 3.5.2 x86
4. Setup Deployment to my GitHub repo master branch.
5. Open Kudu CMD
6. Type d:\home\python35\python.exe -m pip install --upgrade -r d:\home\site\wwwroot\requirements.txt and confirmed that everything works fine.
7. Type D:\home\Python35\python.exe -V and confirmed that Python 3.5.2 displayed.
8. Type D:\home\Python35\python.exe -c "import wfastcgi" and confirmed that output nothing (seems success).
9. Open url. And, 500 <handler> scriptProcessor could not be found in <fastCGI> application configuration error appeared.

New approach 3.5.1 x64

Just curious, I tested with Python 3.5.1 x64 extension that you uploaded (besides 3.5.2 is from Microsoft). It shows 500 but different (and understandable) error occurred.

The LogFiles folder has wfastcgi.log file which not appeared in 3.5.2.
The error message is as follows;

StdErr:
2016-11-04 02:59:03.208505: Unhandled exception in wfastcgi.py: Traceback (most recent call last):
File "D:\home\Python35\wfastcgi.py", line 793, in main
env, handler = read_wsgi_handler(response.physical_path)
File "D:\home\Python35\wfastcgi.py", line 635, in read_wsgi_handler
handler = get_wsgi_handler(os.getenv("WSGI_HANDLER"))
File "D:\home\Python35\wfastcgi.py", line 618, in get_wsgi_handler
raise ValueError('"%s" could not be imported%s' % (handler_name, last_tb))
ValueError: "app.wsgi_app" could not be imported: Traceback (most recent call last):
File "D:\home\Python35\wfastcgi.py", line 602, in get_wsgi_handler
handler = import(module_name, fromlist=[name_list[0][0]])
ImportError: No module named 'app'
2016-11-04 02:59:03.208505: Running on_exit tasks
2016-11-04 02:59:03.223345: wfastcgi.py 3.0.0 closed

Wow, now we can confidently say that the FastCgi module is invoked and the only problem is from my directory setup. The Detailed Error Information (htm file) is different also; the 3.5.2 gives 0x80070585 while 3.5.1 has 0x00000000. So we can assume that two versions do not treat handler setup in same manner.

Approach 3.5.2 x64

• Same as x86 3.5.2. It does not create wfastcgi.log file.

[zooba]

Ah, excellent. I guess the IIS "could not be found" message actually means "it returned an error". In this case, it appears to be crashing. I'll see whether I can reproduce it.

Until then, you may want to try correcting the WSGI_HANDLER variable in your web.config so that it can be found (I think it should be FlaskWebProject1.app in your case) and perhaps that will work fine with 3.5.2.

[YoungjaeKim]

Hi @zooba
I confirmed that 3.5.1 works great with FlaskWebProject1.app editing as you mentioned. So great finally.

However, 3.5.2 x86 and x64 both give same error as below.

scriptProcessor could not be found in application configuration

You can reproduce with my final repository that I tagged.

Could it be marked as a bug? 🐛

[zooba]

Sure - feel free to file it over at https://github.com/Azure/azure-python-siteextensions

[huguesv]

VS 2017 no longer requires a virtual env. Python is installed via site extension.