New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Need straightforward way to honor the caller's preference-variable values / inherited common parameters in functions defined in script modules #4568
Comments
Module has its own context. It may be preferable to set the preference variable value for the module |
Yes, modules have their own scope, which is generally a good thing. However, this special case calls for a concise, PS-supported way to copy all the preference-variable values from the caller's scope. Consider the user's perspective: If Giving module developers an easy, built-in way to opt into the caller's preferences would mitigate that problem. |
Main module's function is to isolate code that expose public API and hide implementation. We can get unpredictable behavior if we just copy any variables (and maybe indirectly functions) to a module context. Remove-Variable ErrorActionPreference
$ErrorActionPreference =@() We may have problems if these variables are simply copied in a module context. What scenarios do we need to? It seems it is only debug. If so we could resolve this otherwise, for example, by loading a module in debug mode. If we want to manage the preference variables in a module, we must do this in a predictable way (We can load tens modules in session!): Import-Module testModule -SetErrorActionPreference Stop |
Also we have an Issue with the request to share modules between runspaces. If we want implement this we can not simply copy any variables - no single parent context exists. |
Indeed, but that is (a) incidental to the issue at hand and (b) should in itself be considered a bug: see #3483 |
We're not talking about any variables. We're talking about the well-defined set of preference variables - even though, as stated, there's currently no programmatic way to enumerate them. |
This is very much a production issue, unrelated to debugging: Currently, if you create a script module, that module's functions will by default ignore a caller's preferences variables (unless you're calling from the global scope), which to me is self-evidently problematic: For instance, if a caller sets |
Honoring the callers preference might cause serious problems by introducing exceptions (e.g. Some examples - If the goal is to silence a noisy function - one can work with the function author to provide a better experience or else explicitly pass |
I think the focus is too much on the implementation here and not on the problem: a user shouldn't have to know or care how a particular PowerShell command is implemented. Whether it's a cmdlet, function, cdxml, imported from a remote session, whatever: they should have a consistent experience with regard to common parameters and preference variables. If I put
I don't see this as a problem, since that's exactly what happens if the caller specifies |
@dlwyatt: Amen to that. I was in the middle of composing the following: The point is that if compiled cmdlets respect preference variables set in the caller's scope, it's reasonable to expect advanced functions to respect them too - the user shouldn't have to worry about module scopes or how a given command happens to be implemented. With respect to Let's take a more innocuous example: setting The following example defines
# Define compiled cmdlet Get-Foo1 (via an in-memory assembly and module).
Add-Type -TypeDefinition @'
using System;
using System.Management.Automation;
[Cmdlet("Get", "Foo1")]
public class GetFoo1 : PSCmdlet {
protected override void EndProcessing() {
WriteVerbose("foo1");
}
}
'@ -PassThru | % Assembly | Import-Module
# Define analogous advanced function Get-Foo2 via an in-memory module.
$null = New-Module {
Function Get-Foo2 {
[cmdletbinding()]
param()
End {
Write-Verbose("foo2");
}
}
}
$VerbosePreference = 'Continue'
# Compiled cmdlet respects $VerbosePreference.
Get-Foo1
# Verbose: foo1
# Advanced function in script module does NOT, because it doesn't see the caller's
# $VerbosePreference variable.
Get-Foo2
# (no output) Expecting the user to even anticipate a distinction here and to then know which commands are affected based on how they happen to be implemented strikes me as highly obscure. A scenario in which the behavior is even more obscure is with implicit remoting modules that proxy compiled cmdlets that execute remotely. In that case, the remotely executing cmdlets, even though they normally respect the caller's preference, do not. (On a related note: even I don't know what the right solution is, but if we can agree that there is a problem, we can tackle it. |
Keep in mind one big difference between binary cmdlets and functions - binary cmdlets are rarely implemented in terms of PowerShell functions or other cmdlets. The error or verbose output is likely carefully crafted for a binary cmdlet, whereas it's a bit more random for functions. Also note that the equivalence of It's worth pointing out that extra verbosity is not always desirable - and this proposal could turn some useful verbose output into noisy useless output. |
Users needing to be aware of how a given command happens to be implemented is an unreasonable expectation.
PowerShell has evolved toward making the PowerShell language a first-class citizen with respect to creating cmdlets (advanced functions).
If I, as a user, set Note that backward compatibility is a separate discussion - opting in to copying the caller's preference variables is a viable option in that context, perhaps via a new
Again I don't fully understand your comment; at the risk of going off on a tangent: They are not equivalent: |
I think you're emphasizing my point. Users of a command aren't aware of the implementation and do have the expectation of reasonable output. Command authors are a different story - they need to be aware of the implications of preference variables and parameters like In some cases, the command author and user are the same, and I do wonder if that's where some of this feedback is coming from - because the command author sometimes wants more help debugging during development. |
No argument there. And let's not forget predictable behavior, which brings us to the next point:
I can't speak for @dlwyatt, but to me this is all about the user perspective:
If I can't predict which of the commands I'll invoke will actually honor the preferences, I might as well do without preferences altogether and only use common parameters. We've covered If I set these with the expectations that all commands invoked from the same scope will honor them, I may be in for nasty surprises. Similarly, to return to |
I think I now understand what you're saying, and I hope I'm not wasting my breath based on a misinterpretation: It sounds like you're conceiving of preference variables as being limited to the current scope only, without affecting its descendant scopes or the potentially completely separate scopes of commands invoked.
Leaving the syntax and backward compatibilities issues aside, I can see how some preference variables can be helpful as limited to the current scope:
|
I found that specifying
It's probably easier to do that with VerbosePreference than adding |
It seems to me that when considering code inside a script module there are two distinct kinds of code to which preference variables (eg.
The above points aren't hard and fast, but I think they're a reasonable starting point for most modules involving a mix of business logic and calls made to outside the module. PowerShell, of course, does not behave this way by default. To implement a module that is consistent with the above points requires two things to happen as follows:
I suspect that these tasks can be handled reasonably well by a utility module. I wrote an annotated proof-of-concept of such a module. The entry points to a user module that uses the utility module would look, for example, like this: function Get-MyItem {
param(
[Parameter(Position=1)] $Path
)
HonorCallerPrefs {
Get-MyItemImpl $Path
}
}
function New-MyItem {
param(
[Parameter(Position=1)] $Path,
[switch] $WhatIf,
[switch] $Confirm
)
HonorCallerPrefs {
New-MyItemImpl $Path
}
} The corresponding sites that call out of the user module would look like this: InvokeWithCallerPrefs {
Get-Item $path @CallerCommonArgs
}
InvokeWithCallerPrefs {
New-Item $path @CallerCommonArgs
}
|
The recent #6556 is a more pernicious manifestation of the problem discussed here: Because CDXML-based cmdlets are seemingly advanced functions rather than binary cmdlets, the functions in the Revisiting @lzybkr's comment:
From what I gather, PowerShell implicitly translating a common parameter such as @alx9r: As commendable as starting scripts / functions with |
#6342 is a similar manifestation of the problem, in the context of |
I hit this problem writing my first module. I'm going to revert back to sourcing scripts directly ( 9+ years for a problem of this impact seems like a long time. Are advanced-function modules considered second-class citizens in PowerShell? Are developers writing binary modules instead? Or is there just very little developer involvement at the module level in general? |
@mc-ganduron most likely the issue is complexity of implementation vs. relatively easy workarounds (passing the preference thru to child functions). I would say abandoning advanced functions for just this is a little dramatic don't you think? |
Not at all. First, I have a prior solution available with no workarounds required (sourcing scripts directly). Second, it's reasonable to take first impressions into account. My first experience with advanced-function modules led me to a 9+ year-old bug. I'm not willing to march ahead into other old bugs when I have options available, within PowerShell and without. |
@mc-ganduron - All software has bugs. Lots of them are old. This one bothered me as well until the day I was troubleshooting a problem with my module and I only wanted to have verbose output from my module and did not want all of the verbose output noise from all of the other modules and functions my module called. This current way does not seem as convenient or consistent at first, but it does represent an "opt-in" style choice that is more useful than opting out for many troubleshooting scenarios. |
Actually I noticed that for errors $null = New-Module {
$ErrorActionPreference = 'Stop'
function Foo {
[CmdletBinding()] param()
trap { $PSCmdlet.ThrowTerminatingError($_) }
Get-Item /Nosuch
}
}
$ErrorActionPreference = 'SilentlyContinue'
Foo I think that it's the most reliable way to handle errors from modules. |
I do wish the syntax for $PSCmdlet.WriteError and ThrowTerminating error was more terse, it looks like black magic when it really should be the way to do it, something like |
I have been scratching my head for months why I didn't get my Debug, Verbose and Information messages in my output when calling Functions in Modules. It's funny when they advise us to move from Write-Host to Write-Information and then make InformationPreference SilentlyContinue the default. By accident I found this thread and I now prefix DebugPreference, VerbosePreference and InformationPreference with $Global and suddenly I get everything I've been missing, all the feedback I had programmed into my functions but could only see when running from my local system. So this is absolutely still a thing on PowerShell 7, Mr. Bot. |
This is so ugly. I want my debug messages and have no problem getting them. Somehow a module gets the preference. I try as I might to disable the module messages, but to no avail. I don't know if it's the module code trying to get the preference via some tricky means because of this issue or something else. Really, perhaps leave this as version 1 of messaging and create a completely new implementation. And make the new implementation easy to use. A transcript option would be nice if we don't want to liter code with write-debug when we want to see everything. A message level instead many preferences. Multiple targets. Bite the bullet and think of something both easy and useful. Don't make it so we are stuck with backward compatible horrors. I'm sick of this thing. |
I don't remember how many ways I tried to suppress module debug messages without suppressing mine. (I'm debugging my code, not the modules.) Here is another attempt that seems to work. (I'm holding my breath.) In the function I define a private set of preference variables if they are different from the desired module preferences. The function will use the private set. A global set of preferences are defined - after the private set is defined - with the desired module preferences. Before exiting the function, the global values are returned to their original values. I'm experienced with PowerShell, but I in no way consider myself an expert. This seems fragile and clunky. Constructive criticism very welcome. (If you don't think I know what I'm doing, that probably makes two of us.)
Note: The function is incomplete. My desire is to run the function and magically check or set the complete database mail configuration on 50-100 SQL instances to conform to standards. In my opinion, if it's very hard to find a good template or pattern for solving a problem, the tool likely has an issue. This is the case for T-SQL and stored procedures. Perhaps PowerShell is not as bad, but I'm not so sure in this case. Also, common problems such as displaying values for debug could have a common template. I'm not fond of trying to create a format for messages. Wish there was an expertly defined format that provided natively. UPDATE: The use of $PSCmdlet.SessionState.PSVariable.GetValue will get the global value. It might be okay to force the use of the global values to make sure they are used. Definitely a design choice that adds a lot of code. Is it just a precaution from getting the incorrect preferences? It won't protect from temporarily setting global preferences and failing to set them back. |
Please reopen this. Looks like the bot closed it. (Sweeping it under the carpet?) |
After that lengthy example, I have to wonder if all I needed to do was like below. In this case Set-StandardDatabaseMail would not have the "hack" code to set the private and global preference values. I don't know if it would handle Debug or Verbose switches as desired. I wish I knew the right way to handle this. If there is no fix, then perhaps a detailed document that explains how we should handle this?
If I was using several modules and want to control them individually, this could become very cumbersome. Should preference variables be private by default scoped by fn and module? UPDATE: It does not work. Sort of expected as private makes it unavailable in the function. Global, Script, and Local scopes appear to be the same for me at this location. Any scope advice? |
There might be an easy workaround if a related VSCode "bug/feature" is fixed/added. PowerShell/vscode-powershell#4327 VSCode dot sources scripts when run in the development environment using F5, so there is no separate script scope. However, when running from the command line, it is possible to not dot source. PowerShell ISE also dot sources scripts, so VSCode is consistent - for good or bad. There does not seem to be a direct way to test not dot-sourcing via VSCode and F5. (Is VSCode is forcing dot sourcing as a best practice by not allowing otherwise?) If dot sourcing can be avoided, the global scope can be reserved for modules while script scope can be used for user script. Then the module preference "firewall" can be useful - even if still wonky. (Introducing a bad practice to compensate for issues controlling module preferences?) Another consideration - must scripts work by design for both dot sourced or not dot sourced use? Perhaps there should be a script level "requires" to force the correct loading of the script? Or perhaps some variables can be forced into a script scope even if the script is dot sourced? Anyway, here is a test script. It demonstrates that running code in VSCode/ISE produces different results from the same code run without dot sourcing.
The first two results show that both VSCode and Windows PowerShell ISE dot source by default when run. Note: the script file is open and run via F5.
It the script file is run without dot sourcing via command line, the results are different.
So, it appears I need to not dot source my first file and then dot source the rest from that file. I suppose this is a VSCode nightmare to support natively. Perhaps two four options in VSCode - always dot source (current), never dot source (user must do it), dot source into a specific file (user sets once), or dot source into a single imaginary file (VSCode uses a hidden file script scope and dots to it). Head can start to hurt thinking about the complexity. |
Here's a trick to create a new scope if the file was dot sourced. It's not the script scope, but might be okay for a simple call.
|
So many years and no change - we are still struggling with this. mklement0 was pretty spot on from the beginning, and it is hard to grasp the perspectives that has been put onto this, attempting to justify inconsistent behavior with user vs developer perspectives. We're not developing to match everyones expectation, but to produce consistent outcomes and predictable behavior. Imho - whether you are looking at this from a user perspective, or developer perspective, either one wants consistent behavior, independent of the source of the function thei're invoking. By having a requirement for detailed knowledge of the call stack from start to end, to be able to predict the outcome of preferences, does exactly complicate it for users and developers alike, and especially -WhatIf is a problematic preference variable/common parameter, since it can incur devastating consequences, which ultimately can't be predicted in the current state of affairs, if implemented in an advanced module function. Keep in mind that in module development, we often call other functions in the module, to deal with specific tasks a greater function needs to accomplish, but at the same time the user/another developer may have the option to call those functions directly. Users and developers alike would expect e.g. -WhatIf to perform as intended, regardless of how deep the function call is in the call stack. It could be catastrophic if we forgot to import preference variables from the callers scope, and even if we do remember it, the inheritance may have been severed by a module function somewhere else in the call stack that didn't do it - which btw. is completely out of our control, and not fixable by an opt-in change. This leaves the WhatIf parameter risky, and thereby imo useless in a module advanced function, as you cannot guarantee its propagation from parent scopes. Propagating preference variables to descendent scopes should always have been the way, but the proposed option of adding a CmdletBindingAttribute for opting in to this, is however a truly elegant solution, that would solve part of the problem, and it won't even introduce breaking changes. You may even add more attributes, for more granular control - so one may wonder what is wrong with at least giving us this? |
@SteveL-MSFT please reopen as active issue. |
In my modules I have found that you can parse the command that called your module load so you can take action on that.... This is a quick sample of how to do it, in case you are like me and need to see code to understand what these words mean. Drop the following code into a module and when you load it use Verbose switch on the import, it should let you know... you can play with it as you see fit. But I was thinking that it should be able to actually implement this in the import command somehow. I wonder if a proxy command would work... you could do it and throw the parameters that you want into a set order for the 'args' array.... or even modify the bound parameters I guess to do it... thats an idea I haven't thought about... I'll go have some fun with that idea this weekend. Anyways, the code: Code
Edited commentI just wish I could make it cleaner and more compact that it would not be so distracting at the top of a module. I also wished that I could get this to format better in the view, but it doesn't for me. Update on findingsYou can actually compute this and then make proxy commands for Write-Verbose and Write-Debug to inject the 'Verbose' or 'Debug' respectively into the PSBoundParameters collection. |
So I am thinking that I might be able to determine if the write-debug or write-verbose is being called from either the module or from a script/cmdlet/function... I might, if that is the case I might be able to build some proxy commands that will do the magic. |
This was closed as 'completed' in Nov 2023, but seems to still be an issue. It's frustrating having to work around this issue with:
everywhere. |
This is a longstanding issue that @dlwyatt explained in detail in this 2014 blog post, and he has even published module
PreferenceVariables
with advanced functionGet-CallerPreference
to ease the pain, via a hard-coded list of preference variable names.In short: A script module's functions do not see the preference-variable values set in the caller's context (except if that context happens to be the global one), which means that the caller's preferences are not honored.
Update: Since implicitly setting preference variables is PowerShell's method for propagating (inheriting) common parameters such as
-WhatIf
, such parameters are ultimately not honored in calls to script-module functions when passed via an advanced function - see #3106, #6556, and #6342. In short: the common-parameter inheritance mechanism is fundamentally broken for advanced functions across module scopes, which also affects standard modules such asNetSecurity
andMicrosoft.PowerShell.Archive
.From a user's perspective this is (a) surprising and (b), once understood, inconvenient.
Additionally, given that compiled cmdlets do not have the same problem, it is not easy to tell in advance which cmdlets / advanced functions are affected. In a similar vein, compiled cmdlets proxied via implicitly remoting modules also do not honor the caller's preferences.
From a developer's perspective, it is (a) not easy to keep the problem in mind, and (b) addressing the problem requires a workaround that is currently quite cumbersome, exacerbated by currently not having a programmatic way identify all preference variables in order to copy their values to the callee's namespace - see Improving the discoverability of PowerShell variables #4394.
A simple demonstration of the problem:
Desired behavior
No output.
Current behavior
Environment data
The text was updated successfully, but these errors were encountered: