New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AutomationNull Behaviour #9997
Comments
using namespace System.Management.Automation.Internal
[AutomationNull]::Value -is [AutomationNull] returning The current workaround for detecting $null -eq $someValue -and @($someValue).Count -eq 0 As for why there is an occasional need to distinguish between a true A command that produces no output technically outputs While # A true $null is sent through the pipeline.
PS> & { $null } | % { 'here' }
here
# A command with no output technically outputs [System.Management.Automation.Internal.AutomationNull]::Value, which is NOT sent through the pipeline - it is an empty enumeration.
PS> & { } | % { 'here' }
# NO output Also, because the $val = & {}
# Because $val contains [System.Management.Automation.Internal.AutomationNull]::Value,
# the switch statement is effectively ignored;
switch ($val) {
default { 'hi' }
} Related issues:
Just like
|
The AutomationNull is defined in System.Management.Automation.Internal namespace that implies only internal non-public use. We free to change the internal AutomationNull but it make sense only if it is really needed for addressing important scenarios. |
If it is purely for internal use it should never be leaking in the ways described above. However, due to the way it's intended to behave, I'm not sure it can be purely internal and still provide the same utility. |
@iSazonov is correct. AutomationNull is for internal use by the runtime. It exists to distinguish an empty stream from a stream containing null.
All of your examples explicitly reference the type. I would hardly call that a "leak".
What do you think it should do that it currently doesn't do and why do you think that behaviour is useful? For example, this using System.Management.Automation.Internal
[AutomationNull]::Value -is [AutomationNull] does not qualify as useful because it's an internal class. |
Sure, for simplicity of definition I opted to use the explicit class name. @mklement0 offered several other ways to obtain the value. Also, it's not internal. I can reference the type name from PS directly. It might be in the internal namespace, but it's public. Here's another way to get an auto-null: $a = if ($false) { Do-Thing } Contrived? Yes. Easily found in a real application? Absolutely. Even more so in a pipeline which may not return any value. |
It's in the internal namespace: System.Management.Automation.Internal This namespace contains classes and interfaces that must be public for some reason but are not considered part of the public PowerShell API. {master}PSCore (1:1) > $a = if ($false) { Do-Thing }
{master}PSCore (1:2) > $a -eq $null
True
{master}PSCore (1:3) > $a.GetType().FullName
You cannot call a method on a null-valued expression.
At line:1 char:1
+ $a.GetType().FullName
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
{master}PSCore (1:4) > So where does AutomationNull show up in this? |
As Dongbo mentioned in the linked PR, AutomationNull doesn't respect GetType(); it's special-cased in that instance to appear as $a -is [psobject] # true
$null -is [psobject] # false
@($a).Count # 0
@($null).Count # 1 If it's not meant to be part of the public API, then y'all need to figure out how to hide it better. 😉 |
@BrucePay I've had to account for it before with methods that have a parameter typed as $a = if ($false) { Do-Thing }
[type]::GetTypeArray(@(0, $a, 1)) And to be fair, the documentation does not do a great job at explaining that it shouldn't be used (and by that I mean it says that it's imperative that you do use it). You also have to remember the audience, most PowerShell folks won't think twice about the namespace because they aren't familiar with that pattern. Granted, most folks who end up needing Whether or not it was intended to be used, I'd still advise that care be taken when it comes to breaking changes. |
Good points, @vexx32 and @SeeminglyScience. I don't think In other words: it is insufficient for PowerShell to pretend that a type('s singleton) is That difference must be discoverable, and having to resort to obscure and cumbersome workarounds to detect So, yes,
is useful - and that it currently returns Furthermore, the type must be documented in the end-user docs. |
@SeeminglyScience, stumbling upon this again I've noticed bizarre behavior around your example: # This FAILS, as in your example.
$anull = & {}; [Type]::GetTypeArray(@(0, $anull)).Name | Should -Be 'Int32', 'PSObject'
# This SUCCEEDS - single-element array, with just AutomationNull.
$anull = & {}; [Type]::GetTypeArray(@(, $anull)).Name | Should -Be 'PSObject'
# RETRYING the ORIGINAL command NOW SUDDENLY SUCCEEDS.
$anull = & {}; [Type]::GetTypeArray(@(0, $anull)).Name | Should -Be 'Int32', 'PSObject' |
Yeah, looks like a binder bug. There must be a difference in how it approaches converting single item arrays vs more populated arrays. That would explain why the third works, the binder would still be cached. |
Thanks, @SeeminglyScience; I've created #11118. |
This comment has been minimized.
This comment has been minimized.
Are we still tracking anything in the issue or we can close? |
Not sure, but I do want to clarify that my previous idea of:
isn't feasible because then these two examples would react differently: $anull = $null | % {}
$anull | % { 'something' }
# vs
$null | % {} | % { 'something' } |
@SeeminglyScience, do you still think the other part of the proposal - making If so, we'd also have to introduce |
If the goal is to make
Might even be able to get away without any breaking changes by doing this: namespace System.Management.Automation
{
public sealed class AutomationNull : PSObject
{
private AutomationNull()
{
}
public static AutomationNull Value { get; } = new AutomationNull();
}
}
namespace System.Management.Automation.Internal
{
[Obsolete("Use System.Management.Automation.AutomationNull")]
public static class AutomationNull
{
public static PSObject Value => System.Management.Automation.AutomationNull.Value;
}
} Auto null should probably be officially supported before any additional enhancements are made. |
Thanks, @SeeminglyScience - my vote is definitely to do it, even if it's only needed 1% of the time, so users can solve the mysteries discussed above. |
@mklement0 Do you ready to ask PowerShell committee? |
I am, @iSazonov - thanks. |
/cc @SteveL-MSFT for PowerShell Committee conclusion. |
Why? If you want to check for
Overall, when we introduced |
For the reasons outlined above.
That doesn't help with checking, the awkwardness and obscurity of which has been outlined above, due to the non-distinction between # Awkward and obscure test whether $x contains [AutomationNull]::Value
$null -eq $x -and @($x).Count -eq 0
That was true up to v2; v3+ preserves the value, and that mustn't change. PS> $x = & {}; $null -eq $x -and @($x).Count -eq 0
True
However, what I think would be a low-risk change - if feasible (can't speak to that) - is to have the # Technically a PSObject, but presents as PSCustomObject
PS> [pscustomobject] @{} -is [System.Management.Automation.PSCustomObject]
True Therefore, along with implementing PS> $x = & {}; $x -is [AutomationNull]
True # wishful thinking I think that would be sufficient and makes for a nice complement to the pending implementation of |
I agree for the most part. I don't think it comes up anywhere near often enough to really touch it aside from maybe making a actual public version that just essentially points to the pubternal one. The wording there was carefully chosen to convey that but it probably wasn't direct enough. My change proposal is more "if it's decided that it should be done for reason x, then this is how it should be done".
It was at one point, but it's assigned to the variable now. Here's an example of what I assume is the reason why, copied from one of my comments above: # If autonull wasn't saved, these two examples would act differently.
$anull = $null | % {}
$anull | % { 'something' }
# vs
$null | % {} | % { 'something' }
Why pubternal then? Why not just a normal public API?
I'm not sure I necessarily agree with the risk assessment, but I'm not arguing the low value side of that. |
Point of clarity that seems to have been ever so slightly glossed over... you can't check |
Good point, @vexx32, and, in a similar vein, to amend my previous comment: While |
@mklement0 That's actually not a lie. When you create a PowerShell/src/System.Management.Automation/engine/MshObject.cs Lines 544 to 550 in da94afa
|
@SeeminglyScience, but it is still wrapped in a PS> [type]::GetTypeArray(([pscustomobject] @{})).Name
PSObject And speaking of conflation and lies: the conflation of PS> [pscustomobject] @{} -is [pscustomobject]
True # OK, but ....
PS> (Get-Item /) -is [pscustomobject]
True # !! ALSO true, because [pscustomobject] is the SAME AS [psobject]
And let's not forget that with Note that there is already precedent for situationally treating # `[pscustomobject] @{}` is *syntactic sugar* for constructing a PSCustomObject
PS> ([pscustomobject] @{}).GetType().Name
PSCustomObject
# `[psobject] @{}`, by contrast, confusingly creates a [hashtable] that is
# virtually invisibly and uselessly in a PSObject instance.
# This despite the fact that both `New-Object PSObject`
# and `[psobject]::new()` *do* create PSCustomObject instances.
PS> ([psobject] @{}).GetType().Name
HashTable |
Well yeah sure but most things are, most of the time. The other stuff is probably better suited for a new issue. My point is that it's not worth making |
@mklement0 I missed this initially:
In my proposal it's still a true singleton. The same exact object will be returned from both The only possible break I can think of is if there is an obscure branch of code where // Would NOT catch auto null
if (obj.GetType() == typeof(PSObject))
{
return obj == AutomationNull.Value;
}
// This would still work
if (obj is PSObject)
{
return obj == AutomationNull.Value
} If there are any instances of that (ideally there wouldn't be as |
Thanks for the clarification, @SeeminglyScience, I had missed the nuances of your proposal; so it sounds like we needn't worry about the proposed change being risky . This leaves us with the question: Why do it?
To me, it's not about how often the need arises, but about the need to provide a way to discover and make sense of a real-world behavioral difference that would otherwise be inexplicable without insider information (currently, the only high-profile source of this information that I'm aware of is this Stack Overflow question, where @PetSerAl has provided an in-depth answer). As it stands, the conflation of Making the following all work (meaningfully) would be a great enhancement in my estimation:
Parting thought re:
Yes, but it's another leaky abstraction that occasionally peeks from behind the curtain to cause seemingly inexplicable behavioral differences: #5579 |
Correct and by design. The intent was to make So again - why? What significant scenarios depend on actively working with |
What risk? There isn't any significant risk whatsoever, unless you have some specific disagreements with @SeeminglyScience's assessments. I think @mklement0 and prior points in this thread have addressed the why of it quite well enough. 🙂 |
The significant risk of breaking all kinds of things in obscure ways because we've done something to make AutomationNull more visible rather than less visible. As I mentioned before, getting a usable system with AutomationNull took months of bug hunting. And I have yet to hear a credible rational for changing the semanitcs of AutomationNull. In a value/expression content AutomationNull should work exactly like null. In a pipeline, it's purpose is to indicate that the pipeline returned no results which is not the same as null. |
@BrucePay Sure, but it doesn't work "exactly" like null in all cases. Therefore having a reliable and accessible way to distinguish the two is desirable so that you can properly handle it when you need to. Your arguments are hypotheticals that we "might" break something, with very little information . The responses from @mklement0 and @SeeminglyScience as well as the occasional question that crops up on a fairly frequent basis in the community chat channels indicates that having a reliable way to distinguish $null from AutomationNull is worth implementing. Unless you can demonstrate something that the proposed changes will actually break, I'm not sure your concerns can be tested in any way. An untestable hypothesis isn't especially useful as a talking point. |
To add to @vexx32' s helpful comment: No one in this thread has advocated changing the semantics of AutomationNull - the proposed change in no way modifies the existing behavior, it merely enables reflection on a preexisting behavioral difference that has hitherto been undiscoverable without insider knowledge. Anecdotal references to past struggles and insisting on not being personally convinced are no substitutes for rational debate, especially if the points that have been made aren't being addressed. |
We were sure that # 9794 is correct but it was reverted. This experience says that each application area can have its own 'null' and it is probably not worth crossing them as far as possible (even if it raises a lot of questions). |
@iSazonov, #9794 was a different story altogether, and in a sense the opposite (complement) of what is being proposed here: it tried to extend the set of types that are conflated with |
@vexx32 @mklement0 The issue description is based on #9794 and mentioned Please strongly take into account a history which Bruce revealed. |
The history he revealed is that there were a lot of bugs when they made the change that allowed autonull to be saved to a variable. Nothing is revealed there, that's a huge change that pretty obviously increases risk. It's sort of like saying that there was a lot of obscure bugs when PowerShell's parser switched from purely token based to AST based so we shouldn't consider adding an overload to If the argument was "this comes up so infrequently that it's not even worth discussing" I'd be with ya 100%. If instead the argument is "significantly more drastic and involved changes we made 10 years ago caused problems" then I have a hard time seeing that as anything other than off topic. |
I mean mainly the old history and experience:
Our new history and experience is that in #9794 we got married [dbnull], [stringnull] and [null] and then stumbled upon AutiomationNull that was highlighted in the separate discussion. In the history no real scenario presents. So question is - what scenarios do we need to fix to help (1) script writers, (2) binary module developers? what can they not do? what can they not overcome? What annoys them and forces them to make bad workarounds? |
@PowerShell/powershell-committee discussed this. We feel that using |
How often this comes up isn't the point, as argued before.
That wasn't the point either; it's not about using, but about discovering something that you can't help but encounter in real life, without the language giving you the ability to understand why not all apparent Recent case in point: https://stackoverflow.com/q/60515757/45375
What the proposal turned into would not result in any breaking change (as @SeeminglyScience has conclusively argued, aside from his not being personally convinced of the need for a change).
Re-reading the OP, I agree; unfortunately I didn't get around to it in time.
The exact opposite is the case: most users are unaware of
In light of the above:
|
See the comments in #9794 for the full extent of the existing discussion, starting with #9794 (comment)
AutomationNull.Value is sometimes detectable and distinguishable from $null in certain cases, for example:
@SeeminglyScience mentioned a possible way we could have it be handled more closely like
[dbnull]
and[nullstring]
are being handled as of #9794, without losing its current function in the pipeline internals, in #9794 (comment):/cc @daxian-dbw @mklement0
I'm personally in favor of Patrick's solution, as it enables the very clear and concise
$item -is [AutomationNull]
with few downsides (the biggest downside being potential implementation complications, but in my opinion these are probably worth tackling). Interested to hear any further discussion on this! 💖The text was updated successfully, but these errors were encountered: