New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature planning: first class Python support #335

Open
christopheranderson opened this Issue May 4, 2016 · 49 comments

Comments

Projects
None yet
@christopheranderson
Member

christopheranderson commented May 4, 2016

This is a tracking item for First Class Python Support. We'll be tracking the requirements and issues for first class Python support via this item. Feel free to leave comments about features/design patterns.

The Functions team and the Microsoft Python team have engaged in a partnership to investigate first class Python support for the Azure Functions runtime.

Programming Model : Basics

Azure/azure-functions-python-worker#2

Programming Model : Data Types

Azure/azure-functions-python-worker#3

HTTP Triggers and Bindings

Azure/azure-functions-python-worker#4

Execution Context

Azure/azure-functions-python-worker#5

Logging

Azure/azure-functions-python-worker#11

Environment variables

Azure/azure-functions-python-worker#12

Version & Package management

Azure/azure-functions-python-worker#6

Testing/CI

TBD

Samples

Out of scope / Nice to have

1. Entry point decorator

Azure/azure-functions-python-worker#8

2. Access to bindings via context

TBD

Change log

5/5 : Chris - Add basic list of feature areas
1/10 : Asavari - Elaborating on the feature areas and programming model
1/11 : Asavari - Adding info about package management and environment variables
1/18 - Adding Samples
1/19 - Incorporating community feedback
1/23 - Incorporating community feedback (part 2)
1/24 - Adding improvements to context and logger
1/25 - Splitting into smaller issues and moving to the Python worker repo

@christopheranderson christopheranderson self-assigned this May 4, 2016

@christopheranderson christopheranderson added this to the backlog milestone May 4, 2016

@edevil

This comment has been minimized.

Show comment
Hide comment
@edevil

edevil May 5, 2016

Support a requirements.txt file and runtime specification for specifying Python 2 or 3.

edevil commented May 5, 2016

Support a requirements.txt file and runtime specification for specifying Python 2 or 3.

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon May 5, 2016

Member

I would take it further than "2 or 3" and make it be by major.minor version.

Member

brettcannon commented May 5, 2016

I would take it further than "2 or 3" and make it be by major.minor version.

@anthonyeden

This comment has been minimized.

Show comment
Hide comment
@anthonyeden

anthonyeden May 16, 2016

The first step may be to publish some Python-specific documentation and examples.

For the benefit of anyone else trying to do HTTP Functions with Python right now, here's some basic example code I threw together: https://github.com/anthonyeden/Azure-Functions-Python-HTTP-Example/

anthonyeden commented May 16, 2016

The first step may be to publish some Python-specific documentation and examples.

For the benefit of anyone else trying to do HTTP Functions with Python right now, here's some basic example code I threw together: https://github.com/anthonyeden/Azure-Functions-Python-HTTP-Example/

@ryanga

This comment has been minimized.

Show comment
Hide comment
@ryanga

ryanga commented Aug 17, 2016

Add python examples for the Azure Functions DocumentDB bindings: https://azure.microsoft.com/en-us/documentation/articles/functions-bindings-documentdb/

@paulbatum

This comment has been minimized.

Show comment
Hide comment
@paulbatum

paulbatum Nov 29, 2016

Member

Support 64-bit Python.

Member

paulbatum commented Nov 29, 2016

Support 64-bit Python.

@davidebbo

This comment has been minimized.

Show comment
Hide comment
@davidebbo

davidebbo Nov 29, 2016

Contributor

Note that short term, it's possible to use a custom Python version using these steps: https://github.com/Azure/azure-webjobs-sdk-script/wiki/Using-a-custom-version-of-Python

Contributor

davidebbo commented Nov 29, 2016

Note that short term, it's possible to use a custom Python version using these steps: https://github.com/Azure/azure-webjobs-sdk-script/wiki/Using-a-custom-version-of-Python

@rawlukMike

This comment has been minimized.

Show comment
Hide comment
@rawlukMike

rawlukMike Feb 2, 2017

EDITED : NOT TRUE

Solution above is not working anymore. I tried every step but in the end as long as you run inside home\site directory, you will always end up with Python 2.7.8.

Also why Python is so slow? It can takes up to 20s only at importing additional library. I tried virtual env and uploading full directory and running it from C# running cmd: "python.exe parameter".
These should be instant, and yet is so long.

rawlukMike commented Feb 2, 2017

EDITED : NOT TRUE

Solution above is not working anymore. I tried every step but in the end as long as you run inside home\site directory, you will always end up with Python 2.7.8.

Also why Python is so slow? It can takes up to 20s only at importing additional library. I tried virtual env and uploading full directory and running it from C# running cmd: "python.exe parameter".
These should be instant, and yet is so long.

@davidebbo

This comment has been minimized.

Show comment
Hide comment
@davidebbo

davidebbo Feb 2, 2017

Contributor

I just retested that technique and it works fine. Did you make sure to expand the zip as you dropped it in D:\home\site\tools?

The better perf will come with the first class support that this thread is tracking. No ETA, but we will get there.

Contributor

davidebbo commented Feb 2, 2017

I just retested that technique and it works fine. Did you make sure to expand the zip as you dropped it in D:\home\site\tools?

The better perf will come with the first class support that this thread is tracking. No ETA, but we will get there.

@rawlukMike

This comment has been minimized.

Show comment
Hide comment
@rawlukMike

rawlukMike Feb 2, 2017

I am sorry for writing wrong. I tried Python 2.7.13 which have no embeddabled version.
Simply used zipped python with my site-packages.
Is there a way to use latest python 2.7.* with some libraries like pandas, xmltodict, numpy and so on... ?

rawlukMike commented Feb 2, 2017

I am sorry for writing wrong. I tried Python 2.7.13 which have no embeddabled version.
Simply used zipped python with my site-packages.
Is there a way to use latest python 2.7.* with some libraries like pandas, xmltodict, numpy and so on... ?

@brettcannon

This comment has been minimized.

Show comment
Hide comment
Member

brettcannon commented Feb 2, 2017

@rawlukMike

This comment has been minimized.

Show comment
Hide comment
@rawlukMike

rawlukMike Feb 2, 2017

Is there any special step for functions? I managed to install python through site extension gallery, but how can i bind it to functions? Basicly how to translate Configuring Your Site from guide above to functions :). Sorry maybe its a simple dummy question.

rawlukMike commented Feb 2, 2017

Is there any special step for functions? I managed to install python through site extension gallery, but how can i bind it to functions? Basicly how to translate Configuring Your Site from guide above to functions :). Sorry maybe its a simple dummy question.

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Feb 2, 2017

Member

Unfortunately for Functions I don't know.

Member

brettcannon commented Feb 2, 2017

Unfortunately for Functions I don't know.

@davidebbo

This comment has been minimized.

Show comment
Hide comment
@davidebbo

davidebbo Feb 3, 2017

Contributor

@rawlukMike the instructions I liked to shoudl work for any Python version. If you drop Python in the Tools folder it should get picked up because it's on the path.

Contributor

davidebbo commented Feb 3, 2017

@rawlukMike the instructions I liked to shoudl work for any Python version. If you drop Python in the Tools folder it should get picked up because it's on the path.

@rawlukMike

This comment has been minimized.

Show comment
Hide comment
@rawlukMike

rawlukMike Feb 3, 2017

Yep my bad. I used my python folder packed, but forgot msi installer puts python27.dll in system. Did not included it in zip. When I used unpacked but not installed folder from msi it works fine. Sorry for troubles.

rawlukMike commented Feb 3, 2017

Yep my bad. I used my python folder packed, but forgot msi installer puts python27.dll in system. Did not included it in zip. When I used unpacked but not installed folder from msi it works fine. Sorry for troubles.

@rawlukMike

This comment has been minimized.

Show comment
Hide comment
@rawlukMike

rawlukMike Feb 3, 2017

Coping fully working Python 2.7.13 to the \home\site\tools is not working.
It worked for 3.x for some reason.

EDIT: After hour or more it started using Python from \home\site\tools

_@davidebbo You said that site\tools is in the PATH however Kudu show otherwise. It says '\site\deployments\tools' is in the path.
I unzip python in \home\site\deployments\tools (This one is IN PATH).
After that running in KUDU example: \home\python.exe -V started showing new one.
So it should work right?
Except its not.
I made simple Python function:
import sys
print(sys.version)
And it results with 2.7.8. With Python 2.7.13 installed in site\deployments\tools and being visible in Kudu as "primary".

So two questions here :).
1.Why even though KUDU is seeing Python from Path (actually there are two line in PATH pointing to python exe now : D:\home\site\deployments\tools, and D:\Python27), Functions still call old one?
2. Why coping Python 3.5 to D:\home\site\tools is working the first place since this one is not in PATH._

EDIT: I went to lunch and when I was back I saw functions using \home\site\tools Python 2.7.13
It just takes lots of time. My main confusion is multiple mentioning that \site\tools is in Path while kudu showed different. Sorry for troubling you guys. Keep it up with Functions they are fantastic :)

rawlukMike commented Feb 3, 2017

Coping fully working Python 2.7.13 to the \home\site\tools is not working.
It worked for 3.x for some reason.

EDIT: After hour or more it started using Python from \home\site\tools

_@davidebbo You said that site\tools is in the PATH however Kudu show otherwise. It says '\site\deployments\tools' is in the path.
I unzip python in \home\site\deployments\tools (This one is IN PATH).
After that running in KUDU example: \home\python.exe -V started showing new one.
So it should work right?
Except its not.
I made simple Python function:
import sys
print(sys.version)
And it results with 2.7.8. With Python 2.7.13 installed in site\deployments\tools and being visible in Kudu as "primary".

So two questions here :).
1.Why even though KUDU is seeing Python from Path (actually there are two line in PATH pointing to python exe now : D:\home\site\deployments\tools, and D:\Python27), Functions still call old one?
2. Why coping Python 3.5 to D:\home\site\tools is working the first place since this one is not in PATH._

EDIT: I went to lunch and when I was back I saw functions using \home\site\tools Python 2.7.13
It just takes lots of time. My main confusion is multiple mentioning that \site\tools is in Path while kudu showed different. Sorry for troubling you guys. Keep it up with Functions they are fantastic :)

@davidebbo

This comment has been minimized.

Show comment
Hide comment
@davidebbo

davidebbo Feb 3, 2017

Contributor

@rawlukMike site\tools is what's on the path for Functions execution. Kudu uses a different one which is for deployment tools, and is not on the path in the Functions runtime. Anyway, glad it's working for you now!

Contributor

davidebbo commented Feb 3, 2017

@rawlukMike site\tools is what's on the path for Functions execution. Kudu uses a different one which is for deployment tools, and is not on the path in the Functions runtime. Anyway, glad it's working for you now!

@rcarmo

This comment has been minimized.

Show comment
Hide comment
@rcarmo

rcarmo Feb 17, 2017

Member

So I'm going down this particular rabbit hole, and keep bumping my head against the peculiarities of git deployment.

As nice as the portal UI is, I think there ought to be a hard requirement/user story that a Mac/Linux developer should only need to git push source code to the service to get a function going, and libraries be deployed either via Kudu (which is tedious and quirky, but passable) or automatically upon detection of the function config files and a requirements.txt.

I'm also not very keen on the current environment variable approach, since it limits the kinds of data functions have readily available.

There ought to be an import context from azure.functions (or similar) to abstract that away and provide a nicer way to deal with inputs and outputs, somewhat like the current JS API.

Member

rcarmo commented Feb 17, 2017

So I'm going down this particular rabbit hole, and keep bumping my head against the peculiarities of git deployment.

As nice as the portal UI is, I think there ought to be a hard requirement/user story that a Mac/Linux developer should only need to git push source code to the service to get a function going, and libraries be deployed either via Kudu (which is tedious and quirky, but passable) or automatically upon detection of the function config files and a requirements.txt.

I'm also not very keen on the current environment variable approach, since it limits the kinds of data functions have readily available.

There ought to be an import context from azure.functions (or similar) to abstract that away and provide a nicer way to deal with inputs and outputs, somewhat like the current JS API.

@rcarmo

This comment has been minimized.

Show comment
Hide comment
@rcarmo

rcarmo Mar 8, 2017

Member
Member

rcarmo commented Mar 8, 2017

@rcarmo

This comment has been minimized.

Show comment
Hide comment
@rcarmo

rcarmo Mar 9, 2017

Member
Member

rcarmo commented Mar 9, 2017

@quangv

This comment has been minimized.

Show comment
Hide comment
@quangv

quangv Apr 14, 2017

sorry slightly off-topic, but what languages/os are currently a first-class citizen on Azure?

Just from a short Google search, I'm seeing:

.Net
Node.js
Java
Docker
Linux
Windows (assuming)
Open-Source

is anything else a first-class citizen? Or where can I find out that information.


@paulbatum Sorry, I moved my question to StackOverflow here.

quangv commented Apr 14, 2017

sorry slightly off-topic, but what languages/os are currently a first-class citizen on Azure?

Just from a short Google search, I'm seeing:

.Net
Node.js
Java
Docker
Linux
Windows (assuming)
Open-Source

is anything else a first-class citizen? Or where can I find out that information.


@paulbatum Sorry, I moved my question to StackOverflow here.

@paulbatum

This comment has been minimized.

Show comment
Hide comment
@paulbatum

paulbatum Apr 14, 2017

Member

@quangv this issue is not the appropriate forum for such a general question. Please keep this issue focus on Python. Thanks!

Member

paulbatum commented Apr 14, 2017

@quangv this issue is not the appropriate forum for such a general question. Please keep this issue focus on Python. Thanks!

@oribarilan

This comment has been minimized.

Show comment
Hide comment
@oribarilan

oribarilan May 15, 2017

Looking forward to Python support in VS Tools for Azure Functions and of course for Python templates in the Azure portal. Thanks!

oribarilan commented May 15, 2017

Looking forward to Python support in VS Tools for Azure Functions and of course for Python templates in the Azure portal. Thanks!

@Krande

This comment has been minimized.

Show comment
Hide comment
@Krande

Krande May 20, 2017

Hey, could you provide some sample code for python using a blob storage trigger of a specific file and performing some operations on the content (say a simple textfile/xml/json) and return a new/updated version of the file back to the storageblob?

EDIT

Nevermind, I figured it out using the following code:
PyFuncnBlob.zip

Anyways, keep up the good work with Functions and Python!

Krande commented May 20, 2017

Hey, could you provide some sample code for python using a blob storage trigger of a specific file and performing some operations on the content (say a simple textfile/xml/json) and return a new/updated version of the file back to the storageblob?

EDIT

Nevermind, I figured it out using the following code:
PyFuncnBlob.zip

Anyways, keep up the good work with Functions and Python!

@oribarilan

This comment has been minimized.

Show comment
Hide comment
@oribarilan

oribarilan May 20, 2017

@Krande - Did this sample code works for you using VS Tools For Azure Functions? Or do you use this inside the portal?

oribarilan commented May 20, 2017

@Krande - Did this sample code works for you using VS Tools For Azure Functions? Or do you use this inside the portal?

@Krande

This comment has been minimized.

Show comment
Hide comment
@Krande

Krande May 20, 2017

I used the portal for testing the sample code.

Krande commented May 20, 2017

I used the portal for testing the sample code.

@asavaritayal

This comment has been minimized.

Show comment
Hide comment
@asavaritayal

asavaritayal Jan 12, 2018

Adding our current thinking on this. Please feel free to leave comments about features/design patterns. @brettcannon thanks for your contribution and feedback on the first draft!

asavaritayal commented Jan 12, 2018

Adding our current thinking on this. Please feel free to leave comments about features/design patterns. @brettcannon thanks for your contribution and feedback on the first draft!

@Krande

This comment has been minimized.

Show comment
Hide comment
@Krande

Krande Jan 12, 2018

Is python 2.7 no longer supported?

Krande commented Jan 12, 2018

Is python 2.7 no longer supported?

@paulbatum

This comment has been minimized.

Show comment
Hide comment
@paulbatum

paulbatum Jan 12, 2018

Member

@Krande It depends on what you mean by supported exactly. The experimental feature of creating python based functions still exists in functions v1 and it uses python 2.7.8. However as I mentioned above, there are significant performance issues with it and we don't plan to bring that implementation into general availability with full support, etc. This issue tracks implementing python support in functions v2 with an implementation that would have good performance characteristics and can be fully supported. The current plan would be for this implementation to target python 3.6, but I think there still needs to be more discussion and planning around this topic (can we support multiple python versions, etc).

Member

paulbatum commented Jan 12, 2018

@Krande It depends on what you mean by supported exactly. The experimental feature of creating python based functions still exists in functions v1 and it uses python 2.7.8. However as I mentioned above, there are significant performance issues with it and we don't plan to bring that implementation into general availability with full support, etc. This issue tracks implementing python support in functions v2 with an implementation that would have good performance characteristics and can be fully supported. The current plan would be for this implementation to target python 3.6, but I think there still needs to be more discussion and planning around this topic (can we support multiple python versions, etc).

@christopheranderson

This comment has been minimized.

Show comment
Hide comment
@christopheranderson

christopheranderson Jan 12, 2018

Member

We are planning on making heavy use of the async library, which requires Python 3.4 or higher. That informs our current desire to avoid 2.7, but would love feedback on specific cases where 2.7 support would be required and 3.4+ wouldn't work.

Member

christopheranderson commented Jan 12, 2018

We are planning on making heavy use of the async library, which requires Python 3.4 or higher. That informs our current desire to avoid 2.7, but would love feedback on specific cases where 2.7 support would be required and 3.4+ wouldn't work.

@berndverst

This comment has been minimized.

Show comment
Hide comment
@berndverst

berndverst Jan 13, 2018

Member

Pipenv and Python 3.6+ seem like the way to go IMO. For anything else customers will still be able to bring their own containers.

I imagine a lot of people will still provide a requirements.txt instead of Pipfile or Pipfile.lock. In that case I recommend installing packages via pipenv install -r requirements.txt. In the case where either Pipfile and/or Pipfile.lock are provided, pipenv install should do the trick.

Member

berndverst commented Jan 13, 2018

Pipenv and Python 3.6+ seem like the way to go IMO. For anything else customers will still be able to bring their own containers.

I imagine a lot of people will still provide a requirements.txt instead of Pipfile or Pipfile.lock. In that case I recommend installing packages via pipenv install -r requirements.txt. In the case where either Pipfile and/or Pipfile.lock are provided, pipenv install should do the trick.

@Krande

This comment has been minimized.

Show comment
Hide comment
@Krande

Krande Jan 15, 2018

Okay, I believe most of my python 2.7 code should be relatively easy to port to python 3.6. My experience with python 2.7 on azure functions is that once I add packages such as numpy and scipy I see a performance decrease running functions. Would that by any chance be related to the performance issue you mention (paulblatum) in v1? If so and upgrading to v2.0 and using python 3.6 fixes that then I welcome the change!

Krande commented Jan 15, 2018

Okay, I believe most of my python 2.7 code should be relatively easy to port to python 3.6. My experience with python 2.7 on azure functions is that once I add packages such as numpy and scipy I see a performance decrease running functions. Would that by any chance be related to the performance issue you mention (paulblatum) in v1? If so and upgrading to v2.0 and using python 3.6 fixes that then I welcome the change!

@asavaritayal asavaritayal modified the milestones: Unknown, Backlog Jan 15, 2018

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Jan 15, 2018

Member

@Krande I'm not on the Functions team, but based on how they have explained the v2 infrastructure to me I believe it will definitely be improved in v2 as functions will be imported once and so any start-up/import overhead will be cut out.

Member

brettcannon commented Jan 15, 2018

@Krande I'm not on the Functions team, but based on how they have explained the v2 infrastructure to me I believe it will definitely be improved in v2 as functions will be imported once and so any start-up/import overhead will be cut out.

@pragnagopa

This comment has been minimized.

Show comment
Hide comment
@pragnagopa

pragnagopa Jan 16, 2018

Contributor

@asavaritayal - over all looks good.

  • Add sample functions.json that uses type property - "Alternately, you can define the data type using the type property in the function.json configuration"
  • Update table under Context section to use context instead of executionContext
  • Need more details on what different log levels mean. For example, see NodeJS Logging and how to they can surface in AppInsight logs. Also, host.json has a tracing setting we need to use that instead of defining "logging"
Contributor

pragnagopa commented Jan 16, 2018

@asavaritayal - over all looks good.

  • Add sample functions.json that uses type property - "Alternately, you can define the data type using the type property in the function.json configuration"
  • Update table under Context section to use context instead of executionContext
  • Need more details on what different log levels mean. For example, see NodeJS Logging and how to they can surface in AppInsight logs. Also, host.json has a tracing setting we need to use that instead of defining "logging"
@asavaritayal

This comment has been minimized.

Show comment
Hide comment
@asavaritayal

asavaritayal Jan 19, 2018

Issue update: Incorporating community feedback and adding some samples.

asavaritayal commented Jan 19, 2018

Issue update: Incorporating community feedback and adding some samples.

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Jan 22, 2018

Member

One thing I realized is my suggestion to put the main function in a __main__ module might be too "cute" as it would prevent people from conveniently e.g. starting a test instance by pointing Python at the directory that contains your function code. Not sure what others think or what module name I would recommend instead (I know Asavari has suggested index.py to match the Node side), but I'm starting to think my initial suggestion for this was bad.

Member

brettcannon commented Jan 22, 2018

One thing I realized is my suggestion to put the main function in a __main__ module might be too "cute" as it would prevent people from conveniently e.g. starting a test instance by pointing Python at the directory that contains your function code. Not sure what others think or what module name I would recommend instead (I know Asavari has suggested index.py to match the Node side), but I'm starting to think my initial suggestion for this was bad.

@1st1

This comment has been minimized.

Show comment
Hide comment
@1st1

1st1 Jan 22, 2018

A few thoughts/questions:

  1. We can support both async- and sync- functions: if the main() function is synchronous (no async qualifier) we automatically run it in an asyncio thread-pool:

    # Would be run with asyncio directly
    async def main():
        await some_nonblocking_socket_io_op()
    
    # Would be run in an asyncio thread-pool
    def main():
        some_blocking_socket_io()

    This approach gives the user a choice of what libraries they want to use. This way async/await becomes an opt-in.

  2. I agree with @brettcannon that using __main__.py module can be suboptimal. __main__.py has a very special meaning in Python: if an 'abc' package has a __main__.py file, it gets run on python -m abc. So a Python package may have a few __main__.py files in it, that are not intended to be used by Azure Functions. We probably want to pick another name to avoid potential problems here. By the way, is it required to have a function.json file per runnable Python function?

  3. context argument: is it optional? Do we want to auto-detect it? If it's the latter, we can make it a keyword-only required argument:

    # The 'context' argument will be injected automatically
    async def main(my_input: str, *, context: Context):
        pass

    We can do the same for context.logger—decouple it from the Context and pass it as a separate keyword-only argument:

    # 'context' and 'logger' arguments will be injected automatically
    async def main(my_input: str, *, context: Context, logger: Logger):
        pass
  4. Will we have a dedicated Azure Functions user module? That way we could provide functions to capture the current Context object instead of injecting it as an argument:

    import azure_functions
    
    async def main():
        context = azure_functions.get_context()

    The package could also contain helpers and fixtures for writing unittests.

  5. Having an explicit entry point decorator is a neat idea. Maybe this should be the one and only way of marking a Python function as runnable? I.e. it's perfectly fine to have a Python file with many helper functions none of which should be directly runnable:

    import azure_functions
    
    # Private helper function
    def helper():
        return 42
    
    # Directly runnable function
    @azure_functions.export  # or @azure_functions.runnable
    def main():
        return helper()

1st1 commented Jan 22, 2018

A few thoughts/questions:

  1. We can support both async- and sync- functions: if the main() function is synchronous (no async qualifier) we automatically run it in an asyncio thread-pool:

    # Would be run with asyncio directly
    async def main():
        await some_nonblocking_socket_io_op()
    
    # Would be run in an asyncio thread-pool
    def main():
        some_blocking_socket_io()

    This approach gives the user a choice of what libraries they want to use. This way async/await becomes an opt-in.

  2. I agree with @brettcannon that using __main__.py module can be suboptimal. __main__.py has a very special meaning in Python: if an 'abc' package has a __main__.py file, it gets run on python -m abc. So a Python package may have a few __main__.py files in it, that are not intended to be used by Azure Functions. We probably want to pick another name to avoid potential problems here. By the way, is it required to have a function.json file per runnable Python function?

  3. context argument: is it optional? Do we want to auto-detect it? If it's the latter, we can make it a keyword-only required argument:

    # The 'context' argument will be injected automatically
    async def main(my_input: str, *, context: Context):
        pass

    We can do the same for context.logger—decouple it from the Context and pass it as a separate keyword-only argument:

    # 'context' and 'logger' arguments will be injected automatically
    async def main(my_input: str, *, context: Context, logger: Logger):
        pass
  4. Will we have a dedicated Azure Functions user module? That way we could provide functions to capture the current Context object instead of injecting it as an argument:

    import azure_functions
    
    async def main():
        context = azure_functions.get_context()

    The package could also contain helpers and fixtures for writing unittests.

  5. Having an explicit entry point decorator is a neat idea. Maybe this should be the one and only way of marking a Python function as runnable? I.e. it's perfectly fine to have a Python file with many helper functions none of which should be directly runnable:

    import azure_functions
    
    # Private helper function
    def helper():
        return 42
    
    # Directly runnable function
    @azure_functions.export  # or @azure_functions.runnable
    def main():
        return helper()
@asavaritayal

This comment has been minimized.

Show comment
Hide comment
@asavaritayal

asavaritayal Jan 23, 2018

Thanks Brett and Yury for taking the time to think through this. Some notes below -

  1. Agree that we should support both async and sync, to allow flexibility of library and enable straightforward demo scenarios. That said, our docs should emphasize why it's important to use async to get more out of the language worker. I'll add this to the docket for later.

  2. @1st1, @brettcannon - A few options are: main.py or __run__.py . Let me know if one of these seems to resonate more, or you have other suggestions. To answer the second part : yes, we do require a function.json file per Function to define the corresponding bindings and their attributes. In fact, the portal UI lets you edit this file in the 'Integrate' tab beneath each function seperately.

  3. Since logger is going to be the only way to write trace outputs to the console, it makes sense to decouple it from context and provide as a keyword-only required argument.

  4. Open to the idea of having a dedicated module for Azure Functions. We've done this for a few other languages in the past and can discuss the pros/cons.

  5. For consistency across the board, we'd like to keep the entryPoint attribute of function.json. Given the time, I'd love for us to have a decorator and promote it as the primary way to declare an entry point in the script.

asavaritayal commented Jan 23, 2018

Thanks Brett and Yury for taking the time to think through this. Some notes below -

  1. Agree that we should support both async and sync, to allow flexibility of library and enable straightforward demo scenarios. That said, our docs should emphasize why it's important to use async to get more out of the language worker. I'll add this to the docket for later.

  2. @1st1, @brettcannon - A few options are: main.py or __run__.py . Let me know if one of these seems to resonate more, or you have other suggestions. To answer the second part : yes, we do require a function.json file per Function to define the corresponding bindings and their attributes. In fact, the portal UI lets you edit this file in the 'Integrate' tab beneath each function seperately.

  3. Since logger is going to be the only way to write trace outputs to the console, it makes sense to decouple it from context and provide as a keyword-only required argument.

  4. Open to the idea of having a dedicated module for Azure Functions. We've done this for a few other languages in the past and can discuss the pros/cons.

  5. For consistency across the board, we'd like to keep the entryPoint attribute of function.json. Given the time, I'd love for us to have a decorator and promote it as the primary way to declare an entry point in the script.

@1st1

This comment has been minimized.

Show comment
Hide comment
@1st1

1st1 Jan 23, 2018

@1st1, @brettcannon - A few options are: main.py or run.py . Let me know if one of these seems to resonate more, or you have other suggestions. To answer the second part : yes, we do require a function.json file per Function to define the corresponding bindings and their attributes. In fact, the portal UI lets you edit this file in the 'Integrate' tab beneath each function seperately.

If we require a function.json file, then maybe we don't need any special filenames for Python scripts? We can require the user to specify the name of the Python file and the name of the function in it in their function.json file.

1st1 commented Jan 23, 2018

@1st1, @brettcannon - A few options are: main.py or run.py . Let me know if one of these seems to resonate more, or you have other suggestions. To answer the second part : yes, we do require a function.json file per Function to define the corresponding bindings and their attributes. In fact, the portal UI lets you edit this file in the 'Integrate' tab beneath each function seperately.

If we require a function.json file, then maybe we don't need any special filenames for Python scripts? We can require the user to specify the name of the Python file and the name of the function in it in their function.json file.

@rcarmo

This comment has been minimized.

Show comment
Hide comment
@rcarmo

rcarmo Jan 23, 2018

Member
Member

rcarmo commented Jan 23, 2018

@brettcannon

This comment has been minimized.

Show comment
Hide comment
@brettcannon

brettcannon Jan 24, 2018

Member

The idea that @1st1 had about just putting it into function.json works for me. That would let people specify the dotted name to the callable object and thus provide anything that they want to call, including an instance of an object for a specific method, e.g.:

# In mypkg.mod
class MyFunc:
   def main(self, *args, **kwargs):
      ...

main = MyFunc.main

And then in function.json:

...
'function': 'mypkg.mod.main`
...

That way users have more flexibility and the only real requirement is that the object be callable.

Member

brettcannon commented Jan 24, 2018

The idea that @1st1 had about just putting it into function.json works for me. That would let people specify the dotted name to the callable object and thus provide anything that they want to call, including an instance of an object for a specific method, e.g.:

# In mypkg.mod
class MyFunc:
   def main(self, *args, **kwargs):
      ...

main = MyFunc.main

And then in function.json:

...
'function': 'mypkg.mod.main`
...

That way users have more flexibility and the only real requirement is that the object be callable.

@asavaritayal

This comment has been minimized.

Show comment
Hide comment
@asavaritayal

asavaritayal Jan 25, 2018

We're moving this issue to its own repo - https://github.com/Azure/azure-functions-python-worker. For comments / feedback, please find the relevant issue in the new repo.

asavaritayal commented Jan 25, 2018

We're moving this issue to its own repo - https://github.com/Azure/azure-functions-python-worker. For comments / feedback, please find the relevant issue in the new repo.

@sergei3000

This comment has been minimized.

Show comment
Hide comment
@sergei3000

sergei3000 Apr 13, 2018

The links under "Samples" in the opening post are not working

sergei3000 commented Apr 13, 2018

The links under "Samples" in the opening post are not working

@BhaskarDasari

This comment has been minimized.

Show comment
Hide comment
@BhaskarDasari

BhaskarDasari Jul 18, 2018

hi can you support requests module inside functions?

Hi azure function is not recognising the requests module . pasted the error and code please suggest the fix

exception:
File "D:\home\site\wwwroot\test\run.py", line 1, in
import requests
ImportError: No module named requests
.
python code:
import requests
import timeit
import datetime

def run_gomupload():
try:

 start_time = timeit.default_timer()
 cdate =datetime.datetime.now()
 print('Cell Start Time: '+ str(cdate))
    
 # code you want to evaluate
 elapsed = timeit.default_timer() - start_time

 cdate=datetime.datetime.now()

 print('\nCell End Time: '+ str(cdate))

 print('Cell Run Duration: '+ str(elapsed))
 # Clean up resources. This includes the container and the temp files
 #os.remove(full_path_to_file) 

except Exception as e:
print(e)

BhaskarDasari commented Jul 18, 2018

hi can you support requests module inside functions?

Hi azure function is not recognising the requests module . pasted the error and code please suggest the fix

exception:
File "D:\home\site\wwwroot\test\run.py", line 1, in
import requests
ImportError: No module named requests
.
python code:
import requests
import timeit
import datetime

def run_gomupload():
try:

 start_time = timeit.default_timer()
 cdate =datetime.datetime.now()
 print('Cell Start Time: '+ str(cdate))
    
 # code you want to evaluate
 elapsed = timeit.default_timer() - start_time

 cdate=datetime.datetime.now()

 print('\nCell End Time: '+ str(cdate))

 print('Cell Run Duration: '+ str(elapsed))
 # Clean up resources. This includes the container and the temp files
 #os.remove(full_path_to_file) 

except Exception as e:
print(e)

@paulbatum

This comment has been minimized.

Show comment
Hide comment
@paulbatum

paulbatum Jul 25, 2018

Member

@BhaskarDasari - please post your question to stack overflow, msdn forum, or as a separate issue in https://github.com/Azure/azure-functions-python-worker - thanks!

Member

paulbatum commented Jul 25, 2018

@BhaskarDasari - please post your question to stack overflow, msdn forum, or as a separate issue in https://github.com/Azure/azure-functions-python-worker - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment