New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reprioritizing PowerShell support in the Azure Functions runtime #585

Open
KirkMunro opened this Issue Nov 10, 2017 · 31 comments

Comments

Projects
None yet
@KirkMunro
Copy link

KirkMunro commented Nov 10, 2017

The Azure Functions Team is going the wrong way when it comes to PowerShell support. Your wiki docs currently state the following:

For v1 of the AzFx Runtime

  • PS support is experimental (aka not supported) using PowerShell 4.0, with no plans to upgrade to version 5 (not surprising given that you are using a custom Windows build behind the scenes, so upgrading is not trivial)
  • Azure Automation is recommended (full support for running PowerShell scripts, but overkill for many tasks compared to using Azure Functions)

For v2 of the AzFx Runtime (cross-platform support, currently in preview)

  • you have chosen not to bring forward PowerShell support in the form that it is currently in (this is a good thing)
  • you have no clear plans for supporting PowerShell in v2, but this could happen at a later time (this, not so much)
  • generally you will not release any language support in v2 if it doesn't have an architecture that allows it to scale and to support advanced triggers

This paints a pretty poor picture for PowerShell use with Azure Functions, which is really a shame given how useful Azure Functions are and how powerful they can be with PowerShell support in them (although the "support" part can be worked around -- just write C# functions that invoke PowerShell cmdlets).

What will it take to get you to reprioritize this, such that you work with the PowerShell Team to get PowerShell Core support in v2 of the AzFx runtime, hopefully designed and implemented in a much better way than how you implemented PowerShell experimental support in the v1 runtime?

@eosfor

This comment has been minimized.

Copy link

eosfor commented Nov 10, 2017

+1
This will be really useful!

@MSAdministrator

This comment has been minimized.

Copy link

MSAdministrator commented Nov 11, 2017

+1 100%

@christopheranderson

This comment has been minimized.

Copy link
Member

christopheranderson commented Nov 11, 2017

Overall, we're fairly confident that once we move to modules, we'll have a scalable model that doesn't require file I/O for each execution. Advanced data types isn't a requirement for us to support a language - we made that decision explicitly.

We definitely apologize for any lack of transparency. We're currently trying to wrap up the language extensiblility functionality we've been building (Node.js and Java as the guinea pigs). We've not prioritized adding the other languages back in, quite yet, since there's still a bit of work left to make easier to test/support new languages (today, every change to our gRPC contract is 3-4 PRs 😒). There's a general roadmap here: Azure/azure-functions-host#1964. We are starting to get the other language experts engaged to begin design for other languages. We're hoping we can get the PowerShell team themselves to help us make sure it's great (where we hacked the first one together ourselves...).

What will it take to get you to reprioritize this, [...]

These types of issues do help 👍. We need to know there's excitement (though usage numbers alone are good enough for us, tbh) and we want to be able to show the PowerShell team there is a folks from their community that are eager, as well. It would be great to keep getting examples of what you're using (or want to use) PowerShell for in Azure Functions. It would also be great to get any feature suggestions we can include in our design discussions.

To start that off, let me ask: If we stayed on PowerShell 4, but did the other plans we had in mind (move to modules, hot runtime/rewrite to language worker, and improved module management), would this be fine to GA?

@anthonype

This comment has been minimized.

Copy link

anthonype commented Nov 11, 2017

+1

@bergmeister

This comment has been minimized.

Copy link

bergmeister commented Nov 11, 2017

+1 because I am already using the PowerShell functions.

@andylyonette

This comment has been minimized.

Copy link

andylyonette commented Nov 11, 2017

+1

@jeffhollan

This comment has been minimized.

Copy link
Member

jeffhollan commented Nov 11, 2017

Yes just to echo @christopheranderson I imagine part of the concern is the note that says we don’t have plans to bring to v2 - that’s likely too strong of a statement. This week I’ve started working with Chris an others on how we can bring it to v2 the right way. What we don’t have now is an ETA as we do want to partner with PowerShell team in Microsoft to help us build it and we have a few pre reqs on our side to build right tools into platform. But in terms of getting us to prioritize PowerShell we do hear the demand for it and it’s definitely very high on our list (second only to Python in my opinion) and hoping to get work started on this very soon. We can continue to update as we learn more and progress.

@Marc-013

This comment has been minimized.

Copy link

Marc-013 commented Nov 11, 2017

+1

@KirkMunro

This comment has been minimized.

Copy link
Author

KirkMunro commented Nov 13, 2017

@christopheranderson and @jeffhollan Thank you for those additional details. That confirms what I originally suspected. I had been chatting with some peers about what was on the wiki vs what seemed to be happening, but the recent updates to the wiki made me less confident. Based on your comments though, you are on the right path, and that is very promising. It would be better for the community if the wiki was reworded with a much more positive spin on it so that people don't get the wrong idea.

Regarding working with SMEs as you add more languages, I have a high level of interest in seeing much better PowerShell support in Azure Functions, and would be happy to provide guidance and assist in this effort when you are ready to move forward with it.

@christopheranderson, to answer your specific question, if you stayed on PowerShell 4 but pushed forward on the other plans you mentioned, I think that would be fine for me for GA, but I would like to look more closely at what exactly those other plans include and which of the issues identified in this issue would be resolved with the changes you have in mind before completely agreeing. I personally haven't had a need for PowerShell 5 in the Azure Functions runtime yet, and while I'll always advocate for the latest, a fixed/improved version 4 would be my priority.

@SP3269

This comment has been minimized.

Copy link

SP3269 commented Nov 13, 2017

++

Microsoft, please support Piwershell as a first-tier language.

@mi-hol

This comment has been minimized.

Copy link

mi-hol commented Nov 21, 2017

@alexkarcher-msft adding PowerShell to the list of supported languages would allow a whole bunch of skilled Windows admins to move workloads to the cloud using the serverless paradigma.
Targetting pwsh 6.1 would be a great goal :)

@Laskewitz

This comment has been minimized.

Copy link

Laskewitz commented May 14, 2018

@jeffhollan: It's been a while since 11 nov 2017. Do you have an update on this? And maybe even an ETA? Would be really helpful! Thanks in advance!

@jeffhollan

This comment has been minimized.

Copy link
Member

jeffhollan commented May 14, 2018

The PowerShell team started doing some investigation into the work required to make this happen. Last I heard they are still planning to help support us in building it but is a bit more work than they initially hoped to subscribe to our gRPC protocol. Targeting having this out in the coming months but can't commit to a more concrete ETA. Definitely a work item we are actively tracking - @asavaritayal specifically from our side

@wobba

This comment has been minimized.

Copy link

wobba commented Jun 7, 2018

We use small PowerShell functions which work with Office 365 and Teams, and use those functions in Logic Apps, Microsoft Flow and PowerApps. Very handy to write small posh functions compared to writing C# for many scenarios - and easier for clients to maintain afterwards, as most people in IT can comprehend the functions and make changes when needed.

I'd be happy with Posh 4 in v1, but would rather see Posh Core support in v2.

@JustinGrote

This comment has been minimized.

Copy link

JustinGrote commented Jun 21, 2018

@wobba FYI that it is now 5.1 in the current runtime since they switched to 2016 on the backend.

I agree that Posh Core support, especially if it can run on a slim Nano or Linux backend for functions, would be the "smart" way forward.

@KirkMunro

This comment has been minimized.

Copy link
Author

KirkMunro commented Jun 21, 2018

Having worked with PowerShell in Azure Functions for a while now, the biggest problem I have is the Azure Function architecture, with multiple instances of functions run in a single process. It makes the whole notion of serverless go out the window when it comes to PowerShell, because there is no isolation from one Azure Function endpoint running PowerShell to the next if they are in the same Azure Function App. You have to think about whether or not different function instances use different versions of modules, or whether or not those modules do any caching, and if so, make sure they are in completely different function apps so that they are not run in the same worker processes, all of which is contrary to the notion of serverless where you shouldn't need to think about such details. I'm hoping that the Azure Function v2 runtime with container support and .NET core resolves this for me, such that individual invocations of Azure Functions are run isolated from one another in containers so that I don't need to think about any conflicts with modules or issues with caching performed by modules.

@jeffhollan

This comment has been minimized.

Copy link
Member

jeffhollan commented Jun 21, 2018

@KirkMunro what you are describing today are limitations in the implementation of v1 experimental languages like PowerShell. This is one of the main reasons it is still experimental as we know in order to get it GA it needs to have execution isolation similar to what is provided with other languages.

That said discussions for this are progressing. This is very high on our backlog right now and hope to have some deliverable in the coming months. It's too early to commit to anything but engineers are actively engaged on design and discussion of what it would take to bring PowerShell as a "GA-able" implementation for the v2 runtime

@KirkMunro

This comment has been minimized.

Copy link
Author

KirkMunro commented Jun 21, 2018

@jeffhollan Thanks Jeff, that's what I was hoping.

For the v2 runtime, with supported languages like C# today, can it be configured to run Azure Function workloads in container instances without reuse (i.e. where each invocation of a HTTP endpoint would result in the worker running inside of a container that is torn down upon completion, giving true isolation between function invocations)?

@jeffhollan

This comment has been minimized.

Copy link
Member

jeffhollan commented Jun 21, 2018

@KirkMunro likely a separate thread / issue than this one but not really sure I understand the need here for tearing down between executions. This isn't a pattern unique to Azure that we keep an instance around between executions. You aren't required to share any state between executions, but you can (and often is hugely useful like being able to leverage an HttpClient or Sql Connection across multiple executions without having to instantiate from scratch every time). If using Kubernetes you wouldn't deploy and destroy a pod for every request, I don't know why you'd want that with functions either (but again this isn't the right issue for this conversation - you'd likely need to find/open another one)

@JustinGrote

This comment has been minimized.

Copy link

JustinGrote commented Jun 22, 2018

@Katli95

This comment has been minimized.

Copy link

Katli95 commented Jun 28, 2018

+1 yeah this is definitely something me and my team would greatly appreciate being carried forward and enhanced for the v2 runtime!

@eosfor

This comment has been minimized.

Copy link

eosfor commented Jul 6, 2018

+1 posh core

@amang2205

This comment has been minimized.

Copy link

amang2205 commented Jul 6, 2018

+1.
Python too

@JustinGrote

This comment has been minimized.

Copy link

JustinGrote commented Aug 17, 2018

@jeffhollan Hey Jeff been a couple months, any progress? Is the activity being tracked anywhere? Thanks for all your teams' hard work.

@JustinGrote

This comment has been minimized.

Copy link

JustinGrote commented Sep 20, 2018

@jeffhollan FYI AWS Lambda now supports Powershell Core, so you guys are in catch-up mode now...
https://aws.amazon.com/blogs/developer/announcing-lambda-support-for-powershell-core/

@asavaritayal

This comment has been minimized.

Copy link

asavaritayal commented Sep 24, 2018

Hi everyone! Quick update on this - the Functions v2 runtime is now GA and we'll be using the new language extensibility model of v2 to build support for PowerShell. Please tune into the repo here - https://github.com/Azure/azure-functions-powershell-worker for feature/design discussion and updates.

@vairamsvsjdo

This comment has been minimized.

Copy link

vairamsvsjdo commented Oct 16, 2018

@asavaritayal PS Core 6 is still not supported or option is not available for creation in portal. Also the documentation says it is still experimental. see link below
https://docs.microsoft.com/en-us/azure/azure-functions/supported-languages

@JustinGrote

This comment has been minimized.

Copy link

JustinGrote commented Oct 16, 2018

@vairamsvsjdo as he said it's coming, it's not available yet. It's in active development and you can follow the powershell worker repository for updates. I see daily updates but it still hasn't reached the preview state yet.

@markgossa

This comment has been minimized.

Copy link

markgossa commented Dec 19, 2018

+1. We already have PowerShell guys writing Azure Functions for monitoring so it would be good to have this fully supported in V2 at some point.

@schwaizi

This comment has been minimized.

Copy link

schwaizi commented Feb 15, 2019

Hi @asavaritayal , do you have any ETA for preview in Azure?

@bergmeister

This comment has been minimized.

Copy link

bergmeister commented Feb 15, 2019

@schwaizi In one of the last community calls, @joeyaiello mentioned that he can be contacted for private previews of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment