Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure function python v2 with external packages HTTP Trigger not visible in Azure Portal after deployment #1262

Open
SiddheshDesai06 opened this issue Jun 13, 2023 · 73 comments

Comments

@SiddheshDesai06
Copy link

Refer this SO Thread and my answer - https://stackoverflow.com/questions/76458717/why-is-the-functions-runtime-not-finding-my-functions-when-i-import-certain-pack/76463210?noredirect=1#> The Azure python Function with V2 programming model including external packages and imports in function_app.py works locally, But when I deploy it in Azure the Function Trigger is not visible. If I deploy default HTTP Trigger with python v2 programming model, The trigger is visible after deployment. But not when i add custom imports in the function_app.py and requirements.txt. Refer this image - https://i.imgur.com/CjCnsY8.png but the function files are visible in app files like here- https://i.imgur.com/R3SZWcW.png . I tried the deployment with App service plan and consumption plan both, Still no luck. I also added AzureWebJobsFeatureFlags:EnableWorkerIndexing settings, I also tried to deploy this function as zip using azure cli zip command with required settings to deploy function as zip, I also tried to deploy it with function core tools func azure functionapp publish command and also via VS code still no luck.

Investigative information

Please provide the following:
  • Timestamp:
  • Function App name: siliconfunc311
  • Function name(s) (as appropriate): Http trigger pythonv2
  • Core Tools version:4.0.5148

Repro steps

Provide the steps required to reproduce the problem:

Refer this SO thread and my answer- https://stackoverflow.com/questions/76458717/why-is-the-functions-runtime-not-finding-my-functions-when-i-import-certain-pack/76463210?noredirect=1#comment134830419_76463210

Expected behavior

Provide a description of the expected behavior.

after deployment HTTP trigger should be visible in Azure portal

Actual behavior

Provide a description of the actual behavior observed.

Function trigger is not visible in Functions section of Function tab but files are visible in app files

Known workarounds

Provide a description of any known workarounds.

Contents of the requirements.txt file:

Provide the requirements.txt file to help us find out module related issues.

numpy

pandas

opencv-python-headless

librosa

torch

torchaudio

torch-audiomentations

torchvision

azure-functions

Related information

Provide any related information
  • Links to source

https://stackoverflow.com/questions/76458717/why-is-the-functions-runtime-not-finding-my-functions-when-i-import-certain-pack/76463210?noredirect=1#comment134830419_76463210

# function_app.py

import logging
import azure.functions as func
import numpy as np
import pandas as pd
import cv2
import librosa
import torch
import torchaudio
import torch_audiomentations
import torchvision
from azure.functions import HttpRequest


app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)



@app.route(route="HttpTrigger")
def HttpTrigger(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')

    #code block with imported packages 
    arr = np.array([1, 2, 3])
    df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
    image = cv2.imread('Screenshot (1).png')
    audio_data, sample_rate = librosa.load('sample.wav.mp3')
    tensor = torch.tensor([1, 2, 3])
    transformed_tensor = torch_audiomentations.PitchShift(sample_rate=sample_rate, p=1.0)
    torchvision.transforms.ToTensor()(image)

    name = req.params.get('name')
    if not name:
        try:
            req_body = req.get_json()
        except ValueError:
            pass
        else:
            name = req_body.get('name')

    if name:
        return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")
    else:
        return func.HttpResponse(
             "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
             status_code=200
        )
@VirajVaitha123
Copy link

Did this problem start recently?

I had a v2 programming function app deployed that was working fine, however last night i made some changes and I can no longer see the functions. I've tried everything and attempting to revert to the old code but I can't seem to see what is different.

Maybe Azure have made some changes that have impacted us?

@bhagyshricompany bhagyshricompany self-assigned this Jun 14, 2023
@bhagyshricompany
Copy link

@VirajVaitha123 thanks for informing and can you share the function name,region,invocationid etc.
So I can investigate same at our end.

@cghgreg
Copy link

cghgreg commented Jun 14, 2023

I'm the author of the SO question and wanted to add that I found this issue only applies when deploying to a consumption plan. When I deploy to a dedicated app service plan, the functions load.

@VirajVaitha123
Copy link

@VirajVaitha123 thanks for informing and can you share the function name,region,invocationid etc. So I can investigate same at our end.

Sure

function name : scryptifyf1
region: north europe,
invocationid: (I can't find this)..

@bhagyshricompany
Copy link

@VirajVaitha123 Thanks for input I will update once check

@VirajVaitha123
Copy link

Thanks for looking into this.

Do we know if this problem is one multiple people are facing, and there is nothing we can do to deploy with Python v2 programming model?

@bhagyshricompany
Copy link

@VirajVaitha123 pls share the start time and end time

@MT--
Copy link

MT-- commented Jun 19, 2023

I have been experiencing the exact same issue for over a week - I have tried the different deployment methods, as well as deleting and recreating the Function App and App Service Plan.

@cghgreg
Copy link

cghgreg commented Jun 19, 2023

Only workaround I know is to switch to the V1 programming model.

@bhagyshricompany
Copy link

@gavin-aguiar pls comment

@ghost
Copy link

ghost commented Jun 24, 2023

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

@Laura10101
Copy link

Laura10101 commented Jun 27, 2023

I'm experiencing a similar issue but with two differences:

  1. I'm using a timer trigger rather than an HTTP trigger
  2. I'm getting the error '1 found, 0 loaded" in my logging as opposed to the error displayed in the SO post.

One additional thing I've noticed when I've probed around in my portal is that Genie is reporting the following error:

Worker failed to index functions
Result: Failure Exception: ImportError: cannot import name '_imaging' from 'PIL' (/home/site/wwwroot/.python_packages/lib/site-packages/PIL/init.py). Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 338, in _handle__functions_metadata_request fx_metadata_results = self.index_functions(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 607, in index_functions indexed_functions = loader.index_function_app(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 48, in call raise extend_exception_message(e, message) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 44, in call return func(*args, **kwargs) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/loader.py', line 151, in index_function_app imported_module = importlib.import_module(module_name) File '/usr/local/lib/python3.10/importlib/init.py', line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File '', line 1050, in _gcd_import File '', line 1027, in _find_and_load File '', line 1006, in _find_and_load_unlocked File '', line 688, in _load_unlocked File '', line 883, in exec_module File '', line 241, in _call_with_frames_removed File '/home/site/wwwroot/function_app.py', line 2, in from core.data.training_image_filestore import TrainingImageFilestore File '/home/site/wwwroot/core/data/training_image_filestore.py', line 1, in from PIL import Image File '/home/site/wwwroot/.python_packages/lib/site-packages/PIL/Image.py', line 103, in from . import _imaging as core

That is followed by this error:

No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).

The key thing that strikes me as odd here is that it's failing to import the package from /home/site/wwwroot/.python_pacakages/...

Looking in my storage account, the reason it's failing is obvious - there's no .python_packages file in wwwroot.

After liaising with the OP of this github issue on my own SO post, we're using identical Azure Pipeline YAML definitions. The key thing is we're both running from zip packages, rather than deploying the code directly so I wouldn't expect there to be anything in wwwroot.

Since my function_app.py file imports a load of custom packages (which in turn import PIL), and does so before declaring the function, my thinking is that the imports are erroring about because Python should be looking for them in the package but is actually looking for them in the site root (working directory). Since they aren't there, it errors out so never makes it as far as far as the function app declaration.

Just a guess but given the errors it would make sense.

What to do about that, however, I don't know!

For now it looks like the two workarounds are:

  1. Downgrade to v1 programming model
  2. Swap to a dedicated ASP rather than consumption plan

I followed the first of these options and this resolved my issue.

@ghost
Copy link

ghost commented Jul 1, 2023

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

@Otimkev
Copy link

Otimkev commented Jul 1, 2023

I'm facing the same challenge over here using python. The Function HTTP triggers are not visible in Functions section of Function tab but files are visible in App files.

@ghost ghost removed the no recent activity label Jul 1, 2023
@sid24desai
Copy link

@bhagyshricompany Can you take a look at this issue? I see multiple customers facing this issue?

@ghost ghost added the no recent activity label Jul 7, 2023
@ghost
Copy link

ghost commented Jul 7, 2023

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

@ghost ghost closed this as completed Jul 10, 2023
@MikeJerred
Copy link

This is still an issue, I am seeing it happen on occasion. As of yet cannot figure out a pattern for why it happens or how to fix it.

@ghost ghost removed the no recent activity label Jul 21, 2023
@SiddheshDesai06
Copy link
Author

@bhagyshricompany Can you check this issue? Looks like its a bug, Multiple customers are reporting it?

@ghost ghost added the needs-attention 👋 label Jul 21, 2023
@saikumaru
Copy link

I'm not sure if MS is looking at this issue, which is kind of sad.
I have opened a new issue here to track- #1338

@jsm-NK
Copy link

jsm-NK commented Oct 30, 2023

Still having this issue. Also I'm trying to implement Azure Durable functions as well. I connect with FTP and confirm that the /home/data/SitePackages has been updated after deploying from Github actions. The logs claim a succesfull deployment but no files/functions are detected by Azure. (Python v2)

I'm on EP1:3 service plan. This really is a critical issue to solve.

Allright, so for everyone having this issue i suggest going through all your requirements file.
image

Details:
As soon as you include a module thats already a part of the python base module - your code will fucntion localy, but when you deploy it to azure no files will even show up. This is reproducale, and no error/debuing feedback will be provided when such a mistake has been done. I myself fell for this, as its just normal practice for me to include libraries/modules in requirements out of habbit when using/testing new ones. I suggest this is producint a lot of the issues many people here have.

@charlesjankowski
Copy link

All I have in my requirements.txt is azure-functions, and still getting it.

@vrdmr
Copy link
Member

vrdmr commented Oct 30, 2023

@charlesjankowski - Can you share your app name for us to investigate?

@charlesjankowski
Copy link

@vrdmr I just added you to a repo showing the use case, let me know what else you'd need, thanks!

@ahmadhori
Copy link

Still having similar issue, the deployment works fine from vscode but when I use Github Actions to deploy The functions disappear from azure portal even though the deployment doesn't through any error

@saikumaru
Copy link

Still having similar issue, the deployment works fine from vscode but when I use Github Actions to deploy The functions disappear from azure portal even though the deployment doesn't through any error

please follow this issue here, #1338

@cdcadman
Copy link

I ran into a similar issue today. I added a new http-triggered function to my azure function app and named the argument _req instead of req. It took me a while to figure out that this was my mistake. Looking through some of these examples, I think part of the problem with the v2 azure function coding model is that there isn't a way (that I am aware of) to write tests that simulate how the function definitions are read in by the Function App (the process of importing function_app.py, finding the FunctionApp object, and getting the definitions from it). The example here only tests the main function that is being passed to the FunctionApp object: https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-decorators#unit-testing . On the other hand, if you write a python Azure WebApp (Flask or Django), you can run it locally in your automated tests, and this does a better job of checking that it will work after you deploy it.

@VictorProphetaE
Copy link

I have the same problem, locally the HTTP trigger appears, but after deploying it, it does not generate any error, it simply says 'No HTTP triggers found'. My doubt is whether the problem is in part of the libraries I'm using, which is langchain and huggingface for embedding, since I've already tried using the sentence transformer. I've already tried almost all the possible solutions on stack and other pages here

@davidmaness
Copy link

davidmaness commented Jan 4, 2024

Same problem here. Linux, python, consumption plan using the v2 function model, zip deployment. I cannot find any code errors. Functions found 1, loaded: 0. It started happening out of nowhere after a small code change, but a quick reversion and re-deploy did not fix this. Based on a couple days of investigation it appears that the dependencies are failing to build silently during the deployment process. I haven't caught this because I'm on the Linux consumption plan and I don't have much of a window into what goes on during zip deployment.

I've found that building the dependencies locally in a Docker container during pre-deployment will circumvent the issue.

func azure functionapp publish <app_name> --build-native-deps

I've tested this by confirming that the functions do not trigger when running the same command with --build remote instead of --build-native-deps. Strangely enough, sometimes deploying regularly (via VSCode Azure extension) will work again after doing it this way, but adding the --build-native-deps flag to the .vscode/settings.json file as a PreDeployTask doesn't seem to do anything.

Due to the fact that this is only one of several errors that could cause Azure Functions to not register upon a successful deployment (see @JarroVGIT's demonstration above), this is a failure of the Linux webapp consumption plan by not giving us much of a window into the zip deployment process for the purpose of debugging. The SCM site is nearly useless.

EDIT: In my specific case, updating requirements.txt with the output of $ pip freeze solved all my problems.

@Soggywaters
Copy link

Soggywaters commented Jan 11, 2024

I was able to solve this by including all dependencies my app used in my requirements.txt file. I was able to solve it with the help of @saari-oj 's comment where they only had the import azure.functions as func as the only import.

I then saw that my requirements.txt file only had that dependency listed. When I added my other project dependencies to the requirements file, it worked! My guess is, when deploying from VS code, the tool creates a .zip dir of the project, uploads it to Azure and Azure does a pip install from the requirements.txt.

To update your project dependencies, make sure to update your requirements.txt file by running:

pip freeze > ./requirements.txt

I'm using the v2 model, Python (v3.10) func app.

@davidmaness
Copy link

I was able to solve this by including all dependencies my app used in my requirements.txt file. I was able to solve it with the help of @saari-oj 's comment where they only had the import azure.functions as func as the only import.

I then saw that my requirements.txt file only had that dependency listed. When I added my other project dependencies to the requirements file, it worked! My guess is, when deploying from VS code, the tool creates a .zip dir of the project, uploads it to Azure and Azure does a pip install from the requirements.txt.

To update your project dependencies, make sure to update your requirements.txt file by running:

pip freeze > ./requirements.txt

I'm using the v2 model, Python (v3.10) func app.

Thanks, @Soggywaters, this is what worked for me. Since that's two people in this thread where this worked as a solution, I'm guessing this is a more common reason people are unable to deploy their functions.

@tomasfernandez1212
Copy link

Plus one.

It is very frustrating to debug issues that occur outside of the function definition.

For example, import errors, missing dependencies, or issues with the binding and trigger definitions.

We are essentially flying blind when trying to debug these problems.

Azure Functions should provide more access to logging that can lend a light to these issues.

@aerickson-clt
Copy link

I can confirm people's observations that this is a dependency issue. My code has a dependency on another package I published privately where I forgot to list jsonschema as a requirement, as in setup(install_requires=[<jsonschema goes here>]) in setup.py. Before fixing that issue I commented out parts of my project and reproduced the situations of finding / not finding the azure functions when publishing by importing / not importing the missing dependency to my code (I commented out the private dependency import to get rid of the transitive import).

I also verified that jsonschema was not being built on the remote build.

After adding jsonschema to my current project's requirements.txt file it worked, and of course, I also expect it to work when I fix the install_requires on my privately published dependency.

This explains why pip freeze > ./requirements.txt solves the symptom, but obviously the real issues are that:

  1. A dependency is missing from their project, and it is likely a transitive one.
  2. Azure Functions should you and fail to build/publish when this happens.

@erfannariman
Copy link

erfannariman commented Feb 21, 2024

Why is this issue closed? Can it please be re-opened, this is still a pending issue which apparently lot of people are facing.

@olofattemo
Copy link

Why is this issue closed? Can it please be re-opened, this is still a pending issue which apparently lot of people are facing.

Agree. Can we get a section on troubleshooting python function startup issues in the dev guide:
https://learn.microsoft.com/en-us/azure/azure-functions/recover-python-functions?tabs=vscode%2Cbash&pivots=python-mode-decorators
Autoconfiguration wizardry is fine but we must be able to observe enough of what is happening to troubleshoot it.

@vrdmr vrdmr reopened this Feb 26, 2024
@kjell-van-straaten
Copy link

for me switching to python 3.10 solved everything.

@shaulimaya
Copy link

I am facing it as well.
Where can I find the logs for the failure?

The function is loaded successfully locally, but fails remotely

@olofattemo
Copy link

I was running consumption plan on Linux using Python 3.10. Had to switch to P1 plan to get a proper SSH console to be able to find the logs under in /var/log. No traces or exceptions in Azure monitor and function was in running state. Expectation was that python functions v2 dispatcher Exceptions would be lifted to Azure monitor.

@mgopez
Copy link

mgopez commented Mar 20, 2024

Switched to Python V1 and it fixed my issues. Unfortunate that we're running into issues like this with V2.

@shreyabatra4
Copy link
Contributor

This is high priority for us in the Functions team and we will have more information and a troubleshooting guide in the coming days - thank you all for your patience and feedback!

@rbm-radius
Copy link

Same issue. I have been going mad trying to figure this out.

I tested this as follows:

  • barebones function app with no imports other than std lib and local modules = works locally, http triggers are found when deployed
  • exact same function, only add one third-party library (sanic, picked at random). The virtual environment has sanic installed. The app works fine locally, but when you deploy to Azure using the VSCode extension, no http triggers are found

@shreyabatra4 I understand your team is working on this, but this is catastrophic because it means functions that depend on third-party packages cannot be deployed or updated, at all.

@JarroVGIT
Copy link

Same issue. I have been going mad trying to figure this out.

I tested this as follows:

  • barebones function app with no imports other than std lib and local modules = works locally, http triggers are found when deployed
  • exact same function, only add one third-party library (sanic, picked at random). The virtual environment has sanic installed. The app works fine locally, but when you deploy to Azure using the VSCode extension, no http triggers are found

@shreyabatra4 I understand your team is working on this, but this is catastrophic because it means functions that depend on third-party packages cannot be deployed or updated, at all.

This is my last message in this thread and then I am going to unsubscribe. A ton of people just post 'me too' or seem to not have read anything in this long thread, especially where I think I have done a pretty decent job in describing why you see this behavior here #1262 (comment) .

In your case, you take a extremely non-straightforward python dependency (sanic), don't mention any details about how you deploy, on what OS you work, on what OS you deploy from, what OS you deploy to, what kind of plan you use or with which dependency you deploy (are you using uvloop? or ujson with sanic?). All of which are crucial in solving your issue. It is clear that Azure Functions in this respect does not provide the user experience that we all would have liked to have, but just complaining that it is "catastrophic" doesn't help you much either and frankly might only be counterproductive for the rest of us.

I wish you the best of luck, I just don't want these kinds of posts in my inbox anymore.

@rmazzine
Copy link

AZ Functions will not show the HTTP triggers if something goes wrong.

For this specific issue of not being able to use an external package, I had the following workaround:

A simple and dirty fix is copying all installed packages to the root dir of your application, in other words, in your .github/workflows/<YOUR_WORKFLOW>.yml you can add after you install your packages and before zipping the release:

    - name: Install dependencies
      run: pip install -r requirements.txt

    - name: Copy all package files from the virtual environment to the root directory
      run: cp -r $(python -c 'import site; print(site.getsitepackages()[0])')/* .

    - name: Zip artifact for deployment
      run: zip release.zip ./* -r

The command cp -r $(python -c 'import site; print(site.getsitepackages()[0])')/* . will copy all packages you installed and add them in your app root dir.

Just, please, bear in mind this is not optimal (neither pretty) and if you can have issues if one of the files in your pip env has the same name as one (or more) files in your root.

Good luck! :)

@stock-ds
Copy link

  1. I had the same issue, and found the actual error which was preventing function from starting from by opening MyAzureFunction/Monitoring/Log Stream/File System Logs and deploying the function. I saw an error that imports were broken even though VS code said deployment was "successful" and function did not appear.

  2. Later it still broke, and then setting up an identical venv locally with WSL and starting it threw the actual error which was incompatibilities between windows and linux.

@tolo
Copy link

tolo commented Apr 17, 2024

Perhaps this only applies when you run locally, but I got this error after running func start:

[2024-04-17T17:32:04.277Z] Worker failed to index functions
[2024-04-17T17:32:04.277Z] Result: Failure
[2024-04-17T17:32:04.277Z] Exception: AttributeError: Anonymous

I simply followed the steps as learn.microsoft.com and did some minor edits to the function (and added some packages) before attempting to run.

The fix for me was to change the auth_level enum from func.AuthLevel.Anonymous to func.AuthLevel.ANONYMOUS, i.e.:

@app.route(route="xxxx", auth_level=func.AuthLevel.ANONYMOUS)

I'm rather new to Python, so this may or may not be related to this issue, but I thought I'd post it in case it would help someone.

@Basavaraj852
Copy link

I am facing this issue Its been observed that teh webapp is not working because the pythoin modules are not being loaded in azure webapp post deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests