Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCM_DO_BUILD_DURING_DEPLOYMENT is set to False leading new packages define in requirement.txt are not getting loaded #61

Closed
ma185360 opened this issue Jan 8, 2021 · 19 comments

Comments

@ma185360
Copy link

ma185360 commented Jan 8, 2021

Hi Team,

We found that if we are doing Kudo zip deployment and want to upload new module then we need to set both SCM_DO_BUILD_DURING_DEPLOYMENT and ENABLE_ORYX_BUILD are set to true.

https://docs.microsoft.com/en-us/azure/azure-functions/recover-python-functions?tabs=manual#mitigate-modulenotfounderror

Now, when we do deployments via Github actions it sets both the variables to false automatically and there is no way we can change this behavior.

Logs:
Run Azure/functions-action@v1
with:
app-name: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
package: .
publish-profile: ***
respect-pom-xml: false
respect-funcignore: false
env:
AZURE_FUNCTIONAPP_NAME: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
AZURE_FUNCTIONAPP_PACKAGE_PATH: .
PYTHON_VERSION: 3.6
pythonLocation: /opt/hostedtoolcache/Python/3.6.12/x64
Successfully passed SCM crednetial from new publishProfile
Using SCM credential for authentication, GitHub Action will not perform resource validation.
Sucessfully acquired app settings from function app (SCM)!
Will archive . into /home/runner/work/_temp/temp_web_package_48933186187879785.zip as function app content
Will use api/zipdeploy to deploy (scm credential)
Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
Setting ENABLE_ORYX_BUILD in Kudu container to false
Update using Client.updateAppSettingViaKudu
Response with status code 204
App setting ENABLE_ORYX_BUILD propagated to Kudu container
Package deployment using ZIP Deploy initiated.

@AmrutaKawade
Copy link
Contributor

is it linux consumption app?

@ma185360
Copy link
Author

ma185360 commented Jan 8, 2021

Yes,it is.

@Hazhzeng
Copy link
Contributor

Hazhzeng commented Jan 13, 2021

Hi @ma185360,
This is an expected behavior since you should resolve the Python dependencies in GitHub workflow. Please follow the template here and add the following section to install packages inside requirements.txt into your function project artifact.

    - name: 'Resolve Project Dependencies Using Pip'
      shell: bash
      run: |
        pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        python -m pip install --upgrade pip
        pip install -r requirements.txt --target=".python_packages/lib/site-packages"
        popd  

@ma185360
Copy link
Author

@Hazhzeng I have that step in my workflow. And In the O/p I can see package got installed(CFFI). Here is the output of that step:

Output:
ls -al
echo "env.AZURE_FUNCTIONAPP_PACKAGE_PATH"
echo './.'
pushd './.'
python -m pip install --upgrade pip
pip install -r requirements.txt --target=".python_packages/lib/site-packages"
ls -al
popd
shell: /bin/bash --noprofile --norc -e -o pipefail {0}
env:
AZURE_FUNCTIONAPP_NAME: ITS-APPOPS-EDL-PREP-ADLS-MON-FUNC
AZURE_FUNCTIONAPP_PACKAGE_PATH: .
PYTHON_VERSION: 3.6
pythonLocation: /opt/hostedtoolcache/Python/3.6.12/x64
total 52
drwxr-xr-x 9 runner docker 4096 Jan 14 11:30 .
drwxr-xr-x 3 runner docker 4096 Jan 14 11:29 ..
drwxr-xr-x 8 runner docker 4096 Jan 14 11:30 .git
drwxr-xr-x 3 runner docker 4096 Jan 14 11:30 .github
-rw-r--r-- 1 runner docker 395 Jan 14 11:30 .gitignore
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 .vscode
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 ADX
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 ADX2
-rw-r--r-- 1 runner docker 1417 Jan 14 11:30 README.md
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 TimerTrigger1
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 TimerTrigger2
-rw-r--r-- 1 runner docker 288 Jan 14 11:30 host.json
-rw-r--r-- 1 runner docker 193 Jan 14 11:30 requirements.txt
env.AZURE_FUNCTIONAPP_PACKAGE_PATH
./.
~/work/adls-monitoring/adls-monitoring ~/work/adls-monitoring/adls-monitoring
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/site-packages (20.3.3)
Collecting adal==1.2.5
Downloading adal-1.2.5-py2.py3-none-any.whl (55 kB)
Collecting azure-storage-blob==12.6.0
Downloading azure_storage_blob-12.6.0-py2.py3-none-any.whl (328 kB)
Collecting cffi==1.14.3
Downloading cffi-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (400 kB)
Collecting sendgrid==6.4.7
Downloading sendgrid-6.4.7-py3-none-any.whl (73 kB)
Collecting tabulate==0.8.7
Downloading tabulate-0.8.7-py3-none-any.whl (24 kB)
Collecting azure-core<2.0.0,>=1.9.0
Downloading azure_core-1.10.0-py2.py3-none-any.whl (125 kB)
Collecting cryptography>=1.1.0
Downloading cryptography-3.3.1-cp36-abi3-manylinux2010_x86_64.whl (2.6 MB)
Collecting msrest>=0.6.10
Downloading msrest-0.6.19-py2.py3-none-any.whl (84 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting isodate>=0.6.0
Downloading isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting PyJWT>=1.0.0
Downloading PyJWT-2.0.0-py3-none-any.whl (15 kB)
Collecting python-dateutil>=2.1.0
Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting python-http-client>=3.2.1
Downloading python_http_client-3.3.1.tar.gz (7.2 kB)
Collecting requests>=2.0.0
Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting chardet<5,>=3.0.2
Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting idna<3,>=2.5
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting requests-oauthlib>=0.5.0
Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting oauthlib>=3.0.0
Downloading oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
Collecting six>=1.6
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting starkbank-ecdsa>=1.0.0
Downloading starkbank-ecdsa-1.1.0.tar.gz (12 kB)
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
Collecting azure-functions
Downloading azure_functions-1.5.0-py3-none-any.whl (124 kB)
Collecting pycparser
Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Using legacy 'setup.py install' for python-http-client, since package 'wheel' is not installed.
Using legacy 'setup.py install' for starkbank-ecdsa, since package 'wheel' is not installed.
Installing collected packages: urllib3, idna, chardet, certifi, six, requests, pycparser, oauthlib, requests-oauthlib, isodate, cffi, starkbank-ecdsa, python-http-client, python-dateutil, PyJWT, msrest, cryptography, azure-core, tabulate, sendgrid, azure-storage-blob, azure-functions, adal
Running setup.py install for starkbank-ecdsa: started
Running setup.py install for starkbank-ecdsa: finished with status 'done'
Running setup.py install for python-http-client: started
Running setup.py install for python-http-client: finished with status 'done'
Successfully installed PyJWT-2.0.0 adal-1.2.5 azure-core-1.10.0 azure-functions-1.5.0 azure-storage-blob-12.6.0 certifi-2020.12.5 cffi-1.14.3 chardet-4.0.0 cryptography-3.3.1 idna-2.10 isodate-0.6.0 msrest-0.6.19 oauthlib-3.1.0 pycparser-2.20 python-dateutil-2.8.1 python-http-client-3.3.1 requests-2.25.1 requests-oauthlib-1.3.0 sendgrid-6.4.7 six-1.15.0 starkbank-ecdsa-1.1.0 tabulate-0.8.7 urllib3-1.26.2
total 56
drwxr-xr-x 10 runner docker 4096 Jan 14 11:30 .
drwxr-xr-x 3 runner docker 4096 Jan 14 11:29 ..
drwxr-xr-x 8 runner docker 4096 Jan 14 11:30 .git
drwxr-xr-x 3 runner docker 4096 Jan 14 11:30 .github
-rw-r--r-- 1 runner docker 395 Jan 14 11:30 .gitignore
drwxr-xr-x 3 runner docker 4096 Jan 14 11:30 .python_packages
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 .vscode
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 ADX
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 ADX2
-rw-r--r-- 1 runner docker 1417 Jan 14 11:30 README.md
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 TimerTrigger1
drwxr-xr-x 2 runner docker 4096 Jan 14 11:30 TimerTrigger2
-rw-r--r-- 1 runner docker 288 Jan 14 11:30 host.json
-rw-r--r-- 1 runner docker 193 Jan 14 11:30 requirements.txt
~/work/adls-monitoring/adls-monitoring

But in my Azure function I keep getting Module Not found error. But if I publish the function from VS code it works perfectly.

Error:
Result: Failure Exception: ModuleNotFoundError: No module named '_cffi_backend'. Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py", line 271, in _handle__function_load_request func = loader.load_function( File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 34, in call raise extend_exception_message(e, message) File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 32, in call return func(*args, **kwargs) File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/loader.py", line 76, in load_function mod = importlib.import_module(fullmodname) File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/site/wwwroot/ADX/init.py", line 6, in from azure.storage.blob import AccessPolicy, BlobServiceClient, ContainerSasPermissions, PublicAccess File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/init.py", line 10, in from ._blob_client import BlobClient File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_blob_client.py", line 25, in from ._shared.encryption import generate_blob_encryption_data File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/storage/blob/_shared/encryption.py", line 19, in from cryptography.hazmat.primitives.padding import PKCS7 File "/home/site/wwwroot/.python_packages/lib/site-packages/cryptography/hazmat/primitives/padding.py", line 13, in from cryptography.hazmat.bindings._padding import lib

@marcd123
Copy link

marcd123 commented Jan 15, 2021

I also have an objection to this functionality.

We recently setup Github actions to deploy our function app, which deploys successfully, and the new code is on the container - however it seems like old versions of the code are still being used and none of feature updates are available.

We tried deploying manually, as we've been doing for months, and now the same thing happens with that deployment - successful deploy but no visible updates.

Now we are searching for a fix to BOTH our pipeline and manual deployment while avoiding recreating the Function App all together.

It took a long time to even discover these settings were being changed. Maybe if it were mentioned in the docs that these settings will be changed that would save people some time. Additionally, I don't think using a Github Action should have such dramatic effects on the function app settings, causing issues in our other operations. I would honestly prefer a Github Action that just runs remote build, much faster in our case and what we've been doing for a while.

We're on Linux EP2 plan, Python 3.6 function app. Do we need to use the WEBSITES_RUN_FROM_PACKAGE feature to resolve this? Are there any recommendations to change the settings on our KUDU container itself?

@marcd123
Copy link

Changing ENABLE_ORYX_BUILD and SCM_DO_BUILD_DURING_DEPLOYMENT via the Github Action change the settings within our KUDU container, but not on the Azure Portal. Changing the settings from Azure portal had no effect on the KUDU container app settings (api/settings).

For anyone trying to revert their ENABLE_ORYX_BUILD and SCM_DO_BUILD_DURING_DEPLOYMENT on the KUDU container, you can use the API here to manually edit the values, or any other app setting values:

https://github.com/projectkudu/kudu/wiki/REST-API#settings

A simple POST request, authenticated using username and password from your app's publish profile should do the trick.

However, I'm guessing these settings will change back to false on their own if you use the Github action again.

@ma185360
Copy link
Author

Correct. I have tried this and can confirm it.

@marcd123
Copy link

@ma185360 were you ever able to get your function app deploying properly with the functions-action?

Does your code/functionality actually update when you deploy, or is just your modules that are not updating?

@ma185360
Copy link
Author

Yes, it worked properly in beginning until i added a new module in my requirements.txt and things went wrong after that. So it's modules which are not getting updated.

@marcd123
Copy link

Very interesting.

In our case, the deployment is successful, and by viewing files in KUDU we can see python code and requirements.txt are in fact updated in the site root folder. When requesting the function though, we still get the results from old code.

I think this has something to do with WEBSITES_RUN_FROM_PACKAGE. I believe this github action actually prefers that method (at least for Linux apps). You could try setting WEBSITES_RUN_FROM_PACKAGE to 1 (or true, I never know which one is right) in your app settings (on azure portal but also via the KUDU API just in case). Then you might see your changes reflected. I'm going to do the same.

@20shivangi
Copy link
Contributor

Is this issue still being faced ?

@tgdfool2
Copy link

Still facing the exact same issue with the following Workflow definition:

    ...
    - name: Setup Python ${{ env.PYTHON_VERSION }} Environment
      uses: actions/setup-python@v2
      with:
        python-version: ${{ env.PYTHON_VERSION }}

    - name: Resolve Project Dependencies Using Pip
      shell: bash
      run: |
        pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        python -m pip install --upgrade pip
        pip install -r requirements.txt --target=".python_packages/lib/site-packages"
        popd

    - name: Run Azure Functions Action
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
        package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
        publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }}

My requirements.txt file:

# DO NOT include azure-functions-worker in this file
# The Python Worker is managed by Azure Functions platform
# Manually managing azure-functions-worker may cause unexpected issues

requests

Querying the application fails with the following error:

Result: Failure Exception: ModuleNotFoundError: No module named 'requests'. 

Tried with different combinations of ENABLE_ORYX_BUILD, SCM_DO_BUILD_DURING_DEPLOYMENT and WEBSITES_RUN_FROM_PACKAGE... without luck until now :-(

Deploying through VSCode Extension works flawlessly...

Did someone of you have any luck deploying a Python Function in Azure through GitHub Actions with custom requirements?

Thanks in advance for your help!

@marcd123
Copy link

Hey @tgdfool2 I had to use this API to change the ORYX/SCM settings on our VM, changing the settings in Azure portal had no effect, see below.

Keep in mind, the functions-action will automatically change these back whenever it deploys your app, so using this API will just change the settings back so you can deploy another way.

We ended up creating a custom deploy github action. We installed the azure-functions-core-tools in the github action and then used the remote build from there.

Changing ENABLE_ORYX_BUILD and SCM_DO_BUILD_DURING_DEPLOYMENT via the Github Action change the settings within our KUDU container, but not on the Azure Portal. Changing the settings from Azure portal had no effect on the KUDU container app settings (api/settings).

For anyone trying to revert their ENABLE_ORYX_BUILD and SCM_DO_BUILD_DURING_DEPLOYMENT on the KUDU container, you can use the API here to manually edit the values, or any other app setting values:

https://github.com/projectkudu/kudu/wiki/REST-API#settings

A simple POST request, authenticated using username and password from your app's publish profile should do the trick.

However, I'm guessing these settings will change back to false on their own if you use the Github action again.

@tgdfool2
Copy link

Thanks for your answer @marcd123.

Actually, I also tested the "manual" way by using the func binary as part of a GitHub Action step and I was able to deploy my function, including dependencies.

In the end, it turns out that the problem was not related to the deployment tool I used (Azure/functions-action@v1 or func), but it was related to the fact that I was missing the FUNCTIONS_WORKER_RUNTIME App Setting configuration parameter. After having setting it to python, everything seems to work as expected... event with the Azure/functions-action@v1 Action :-)

As reference, please find here my App Settings:
image

With those settings, the Workflow described in my previous post is working like a charm.

Hope this helps other people as well!

@20shivangi
Copy link
Contributor

@tgdfool2 Sorry for asking again.
From your last message, it appears that issue is being faced no more. Can you please let us know if the issue still there ?

@tgdfool2
Copy link

Hi @20shivangi,

As mentioned in my previous answer, after having properly set the FUNCTIONS_WORKER_RUNTIME App Setting, I didn't face the issue anymore. Not sure if other people are in the same situation...

@20shivangi
Copy link
Contributor

@ma185360 Are you still facing this issue ?

@20shivangi
Copy link
Contributor

Since there has been no response, we are closing this issue. Please feel free to reopen in case of any queries.

@brc-dd
Copy link

brc-dd commented Mar 31, 2021

From v1.3.1 one needs to put

    scm-do-build-during-deployment: true
    enable-oryx-build: true

As suggested in release notes, your workflow configuration will look like this:

# Comment out 'Resolve Project Dependencies Using Pip' section as it is no longer needed.
# Python package resolution will be completely handled by remote build
# - name: 'Resolve Project Dependencies Using Pip'
#   shell: bash
#   run: |
#     python -m pip install --upgrade pip
#     pip install -r requirements.txt --target=".python_packages/lib/site-packages"
#
# Set scm-do-build-during-deployment and enable-oryx-build to 'true' in GitHub Action
# to ensure remote build is enabled for this deployment
- name: 'Run Azure Functions Action'
  uses: Azure/functions-action@dev
  id: fa
  with:
    app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
    package: '.'
    publish-profile: ${{ secrets.LINUX_PYTHON_DED_ENABLE_RB }}
    scm-do-build-during-deployment: true
    enable-oryx-build: true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants