Lambda and other thoughts

[daxian-dbw]

• Allow the user to declare assembly dependencies. (what UX?)

  • e.g. A function depends on a custom assembly, which will be loaded by using Assembly ... or Add-Type ..., or Assembly.Load(...). Do we want to support that scenario? One solution would be the AssemblyLoadContext.Default.Resolving.

• Capture Write-Host to logging Write-Host also write the message to Information stream, so we are good here.

• PowerShell based Lambda doesn't bundle AWSPowerShell.NetCore (AWS version of AzureRM.NetCore) with the runtime, but instead, bundle it with user's scripts in the deployment package. The user needs to use #Requires -Modules in script to specify the module dependency.

  • I don't like having AzureRM.NetCore bundle with user's Function App, that will likely cause duplications of such module, which is a waste of bandwidth. However, that would solve the module version issue, since user needs to be explicit about what version of AzRM to be deployed.
  • But we do need a way to let user specify what custom module they want to deploy along with their scripts. (what UX? use #Requires -Modules like Lambda does?)
  • Just realized that the func CLI doesn't have a pack option. Is the deployment restricted to container image? dotnet-lambda package pack the function app into a zip file, ready for deployment.
    • Lambda provides a powershell module to facilitate creating a powershell lambda, including templates, create a package (zip), publish a package to AWS.
    • By having a powershell helper module, it's able to parse the script to get the module dependencies from #Requires -Modules ..., and pack the dependencies into the package. We cannot find the dependency moduels without running in the PowerShell session that the user uses for creating the function.

• Lambda tool is a .NET Global Tool "Amazon.Lambda.Tools". Maybe func CLI should consider to be a Global Tool too, given it's a .NET Core application. (not related to the PS worker)

[TylerLeonhardt]

I like these thoughts. I have a few follow ups.

---

Allow the user to declare assembly dependencies. (what UX?)

Can you give an example of this?

---

I don't like having AzureRM.NetCore bundle with user's Function App

I liked our previous idea of having a Module path at the worker level and a module path a the user level.

$env:PSModule = 'UsersFunctionApp/Modules;workers/powershell/Modules'

If we go this route, then #Requires -Modules makes sense too because it could load the "correct" version if multiple are present.

This lets us ship things in the worker while allowing the user to use another version of a module since importing resolves left to right in the module path.

---

Lambda tool is a .NET Global Tool "Amazon.Lambda.Tools". Maybe func CLI should consider to be a Global Tool too, given it's a .NET Core application.

This should probably be an issue in the core tools repo since it's about the func CLI as a whole.

https://github.com/azure/azure-functions-core-tools

[daxian-dbw]

Allow the user to declare assembly dependencies. (what UX?)

Can you give an example of this?

I updated this bullet to give an example.

[SteveL-MSFT]

func CLI as a global tool makes sense but is orthogonal to this and I suspect many PowerShell scripters won't have dotnetcli installed.

From a packaging standpoint, I think it would be interesting to have a generic way to package a script and dependencies so that ITPros can easily distribute them in their environment even outside of AzF. This can simply be a zip with a well known extension and layout (including assemblies).

I like the idea of requiring the user to package Az module rather than having it in our worker image to resolve the versioning problem.

[TylerLeonhardt]

I like the idea of requiring the user to package Az module rather than having it in our worker image to resolve the versioning problem.

If we go through with this, then we can conditionally call Connect-AzAccount which will speed up start up for function apps that don't need to interact with Azure. That said, then we need to tread carefully so that the user knows how to package their app correctly.

[JustinGrote]

As a user, I am partial to @tylerl0706's plan of having some common base modules like Az in the worker image but "further right" in the PSModule path so we can optionally supply a newer one. It would be the most user friendly and performant without sacrificing ability to bring more "bleeding edge" modules if you need them for certain new features/functionality.

[TylerLeonhardt]

Yeah that would play nice with #76. 😊

[KirkMunro]

A few general thoughts related to this. I've been using a C# Azure Function App that invokes PowerShell for quite a while, and here are the things I've had to deal with along with my principal issues with Azure Functions and PowerShell:

1. Updating PSModulePath (related to Add FunctionApp "modules path" to PSModulePath #76). My function app injects an entry into index 1 in PSModulePath so that I can override any built-in modules (AzureRM in particular) with the latest version that I upload into my function app.

2. Dealing with caching (related to Publish guidance for module authors on how to prepare for multi-runspace or multi-threaded PowerShell use PowerShell/PowerShell#7703). Caching is a huge problem for PowerShell in Azure Function Apps that needs attention. The lack of guidance and poor design of some modules, combined with Azure Function Apps not using process-level isolation, makes for a difficult environment to work with. Technically runspace-level isolation is all that is needed, but that's not even guaranteed with some modules (AzureAD for example). Depending on the setup, if you're working with multiple tenants in some cloud provider or online service, you need to be able to ensure that PowerShell commands don't impact other running instances of the Azure Functions.

3. Supporting SxS versions of binary modules. I'm not sure what is possible here, but this is another issue. If you are running PowerShell through Azure Function App endpoints where different invocations require different versions of the same binary module(s), since everything is in a single process, what can you do? I would love if you could import different versions of the same binary module(s) in different invocations of an Azure Function App endpoint. Process-level isolation gives you this for free, but otherwise, is this even possible in a single process through some modern app domain magic?

For the last two points I don't know what option there is other than looking at Lambda as a good alternative since they have process-level isolation. I've proposed the idea of a multi-process option instead of using multi-threaded workers to the Azure Function team but they don't seem interested in that approach.

[daxian-dbw]

Thanks @KirkMunro for sharing your experiences. For the last two points, it's doable by loading a module to a separate AssemblyLoadContext. Both code and data (statics) from a module can be isolated in that way.
The idea is to add a new flag -Isolated to Import-Module to explicitly make powershell load a module to a separate assembly context. The tricky part is how the type resolution in powershell should change accordingly. I will write up an RFC on this after PSConf Asia, probably in late November.

[KirkMunro]

@daxian-dbw That's great to hear.

In addition to that work, on my second point, above, I think it would also be well worth the time for the PowerShell team to implement runspace-local storage, and that probably merits another RFC. The idea for runspace-local storage is straightforward: provide a thread-safe storage model for PowerShell modules that allows for runspace isolation of cached data in memory. If PowerShell had such a model in place, coupled with design documentation pushing binary module authors to use runspace-local storage for cached information (and, if it's not asking too much also coupled with a PowerShell Module Analyzer or FxCop style rules for Binary modules that red flag the use of static properties for memory caches, but now I'm really dreaming in color), then we wouldn't have to push module authors post-release to fix their in-memory caching logic so that it's designed properly for PowerShell.

[daxian-dbw]

... implement runspace-local storage ... pushing binary module authors to use runspace-local storage for cached information

Binary module authors can already do this today, using ThreadStatic or ThreadLocal object. A ThreadStatic member is stored in the thread local storage, and is isolated for each thread. Runspace itself is also stored in a ThreadStatic member Runspace.DefaultRunspace. So when the same binary module is loaded in another Runspace running in a different pipeline thread, the ThreadStatic members of the module won't carry the existing cache.

[KirkMunro]

I know each runspace has its own thread, but isn't it possible to have multiple threads using a single runspace? In which case ThreadStatic/ThreadLocal wouldn't be appropriate, correct?

[daxian-dbw]

@KirkMunro I get your point. It's more about the PowerShell API. With the same Runspace, $ps.Invoke() will start a new thread every time it's called. ThreadStatic and ThreadLocal would mean no caching for every run in this scenario. This makes the Runspace-local interesting for binary modules.

[joeyaiello]

Next steps to generate work items out of the discussion here.