Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding an automatic variable for the last result of a computation #7853

Closed
Masterxilo opened this issue Sep 24, 2018 · 26 comments
Closed
Labels
Issue-Discussion the issue may not have a clear classification yet. The issue may generate an RFC or may be reclassif Resolution-No Activity Issue has had no activity for 6 months or more WG-Engine core PowerShell engine, interpreter, and runtime

Comments

@Masterxilo
Copy link

Maybe calling it $__.

See https://stackoverflow.com/questions/14351018/powershell-is-there-an-automatic-variable-for-the-last-execution-result

@RichardSiddaway
Copy link

$__ is too similar to $_

We showed how to do this, variable was $last, by wrapping Out-default in a proxy function.

see page 382 of PowerShell in Action, third edition

@mklement0
Copy link
Contributor

mklement0 commented Sep 24, 2018

@Masterxilo: In addition to @RichardSiddaway's approach, the following simpler one may suffice (place it in your $PROFILE):

# UPDATE: USE THE OTHER SOLUTION BELOW INSTEAD.
$PSDefaultParameterValues['*:OutVariable'] = '__'

(Since this is a custom solution, I've stuck with your proposed name, $__, even though as a built-in feature the similarity with $_ may be too confusing, as stated.)

Update: The following variant, courtesy of @vexx32, is preferable in general, and it also avoids the additional processing overhead that the solution above incurs, making the performance/resource-consumption debate below moot. A fundamental limitation, however, is the inability to capture output explicitly sent to Format-* cmdlets.

$PSDefaultParameterValues['Out-Default:OutVariable'] = '__'

For more information, see this answer I've just posted to the Stack Overflow question you link to above.

@BrucePay
Copy link
Collaborator

@mklement0 Your suggestion doesn't save the output of the last command, it saves the output of every single command that gets run! It appears to work because only the last variable binding persists. Doing this means that If you have 3 commands in a pipeline then the output of each stage is saved in an ArrayList. The result is pervasive execution overhead plus additional work for the garbage collector. I would strongly recommend against this approach.

@mklement0
Copy link
Contributor

mklement0 commented Sep 24, 2018

@BrucePay:

To quote from my linked SO answer (update: since rewritten to recommend @vexx32's variant):

Use of this technique slows your commands down, though usually only negligibly so (the more nested pipelines a command comprises, notably ones run in a loop, the higher the performance impact).

In practice, I would argue, this technique works fine in interactive sessions without noticeable performance impact in typical scenarios.

@BrucePay
Copy link
Collaborator

@mklement0 I certainly run a lot of scripts from my interactive environment. In practice, this is a wasteful solution for something that can be achieved much more efficiently and with little effort. I don't recommend it in any scenarios.

@mklement0
Copy link
Contributor

mklement0 commented Sep 25, 2018

@BrucePay:

this is a wasteful solution

My solution is a pragmatic one that is trivial to implement.

As for the wastefulness: My guess is that users won't notice the processing overhead in practice, while still reaping the benefits.

something that can be achieved much more efficiently and with little effort

Do tell us how. If it's what @RichardSiddaway alluded to: it won't be nowhere near as simple.


The above debate is moot, however, if we decide to make this a built-in feature, which sounds like a good idea - even if details need to be worked out (limit on in-memory size, ...).

What do you suggest?

@vexx32
Copy link
Collaborator

vexx32 commented Sep 25, 2018

Does Out-Default support -OutVariable? Wouldn't it be more pragmatic to attach it only there, such that only the final output is stored in the outvariable, rather than every step of a lengthy pipeline?

Insert testing montage here...

... I might actually use this, it works pretty dang well. Easier than proxying the whole function, too!

@iSazonov iSazonov added Issue-Discussion the issue may not have a clear classification yet. The issue may generate an RFC or may be reclassif WG-Engine core PowerShell engine, interpreter, and runtime labels Sep 25, 2018
@mklement0
Copy link
Contributor

mklement0 commented Sep 25, 2018

@vexx32:

Excellent idea - that hadn't even occurred to me.

Aside from incurring less processing overhead, there is a functional difference, however - still, that too could be considered an improvement; depending on your preference, you may choose one over the other:

  • $PSDefaultParameterValues['*:OutVariable'] = '__' saves output in $__ even if it is captured or redirected (say with $var = ... or ... >$null).

    • Update: Aside from the additional processing overhead, this solution falls short in that it captures the formatting objects output by the Format-* cmdlets used behind the scenes rather than the actual data. As an aside: Curiously, using a Format-* cmdlet explicitly captures nothing (returns an empty array list).
  • $PSDefaultParameterValues['Out-Default:OutVariable'] = '__' saves only the objects output to the terminal; as such, what $__ contains isn't necessarily output from the most recently executed command, but the (terminal-bound) output from the most recently executed command that produced terminal output.
    However, that is also how _ appears to work in Python, from what I can tell.

@kilasuit
Copy link
Collaborator

I tend to use this way

Get-Command *-PSRead* | Tee-Object -Variable ou

which gives the output I want to the screen and to the $ou variable

though the disadvantage is that I need to know thats what I want to happen before issuing the command

@vexx32
Copy link
Collaborator

vexx32 commented Sep 25, 2018

@mklement considering that it would be considered similar to $LASTEXITCODE and similar things, I would argue that in most cases, direct console use is the main time you'd want it to take effect -- but as you say, it'd be up to personal preference and use case. 😄

In any case, I'm definitely adding that to my own profile, very handy!

@mklement0
Copy link
Contributor

@vexx32: Agreed on all counts; $PSDefaultParameterValues['Out-Default:OutVariable'] = '__' is probably the best general-purpose choice (no additional processing overhead, and no saving of output that is saved elsewhere anyway (variable assignment, redirection to file) or suppressed on purpose (>$null, ...); and, as stated, it's how Python does it too.

@kilasuit:

though the disadvantage is that I need to know thats what I want to happen before issuing the command

Yes, having output collected automatically has two advantages:

  • you don't have to plan ahead

  • there's nothing additional to type.

@mklement0
Copy link
Contributor

mklement0 commented Sep 25, 2018

@vexx32: Unfortunately, while we can have our cake, we can only eat it partially:

$PSDefaultParameterValues['Out-Default:OutVariable'] = '__' doesn't work if an explicit Format-* call is used (presumably because Out-Default is then not called) - nothing is captured then ($__ is left untouched).
In the case of $PSDefaultParameterValues['*:OutVariable'] = '__', curiously, you get an empty array list.

An unsatisfying workaround is to capture Format-* output in a different variable, which not only requires you to think about which variable you need to target, but you'll still only see formatting objects rather than data, and, since Format-* cmdlets are involved behind the scenes even if you don't use them explicitly, the output of commands without Format-* calls is then captured twice - once as data, in $__, and again as formatting objects, in the other variable.

In short: unless there is something I'm overlooking, it seems that writing an Out-Default proxy function is the way to go for now - and hopefully we'll have a built-in feature soon.

@vexx32
Copy link
Collaborator

vexx32 commented Sep 25, 2018

You know, that doesn't even really bother me that much. Most of the time, if I pipe to the Format-* cmdlets, I don't want to reuse that output -- if I did want it, I'd have to rewrite the line anyway to remove the format command.

Once again... very little lost. :D

@mklement0
Copy link
Contributor

@vexx32: Got it.
P.S.: I hadn't noticed that my original approach, $PSDefaultParameterValues['*:OutVariable'] = '__', actually captures formatting objects rather than data, so it's of no use for that reason alone; I've also updated my SO answer to recommend your approach instead.

@vamolessa
Copy link

From my understanding, $PSDefaultParameterValues['Out-Default:OutVariable'] = '__' does not capture output from other executables (non-cmdlets).

Is there a worksaround? Or entirely impossible even by changing powershell itself?

@vexx32
Copy link
Collaborator

vexx32 commented Dec 4, 2019

I'm sure it's possible. I'm not fully sure why calling an external executable bypasses Out-Default entirely but I'm sure it's probably deliberate; Out-Default is where PS data is sent, but external executables likely don't need that, since in a majority of cases the output is simple unstructured text.

@mklement0
Copy link
Contributor

@vexx32, I assume it's related to the by-design behavior of passing external programs' output streams through to the console (unless captured or redirected), enabling programs such as vi to work normally.

function Out-Default { Write-Host 'here' } # All PowerShell commands will now print just 'here'

whoami   # external program: Out-Default is NOT called

# An explicit redirection routes the output through PowerShell's streams and 
# therefore calls Out-Default,
# but *still passes the output through*(!) as well.
whoami *>&1

@mklement0
Copy link
Contributor

mklement0 commented Dec 4, 2019

I should add that *>&1, even though it causes Out-Default to be called, is actually ineffective as a workaround (and 1>&1 fundamentally doesn't work).

Currently, the simplest (but still cumbersome) workaround is:

whoami | Write-Output  # $__ is now populated.

@adam-c-anderson
Copy link

echo is a built-in alias for Write-Output.

whoami | echo

isn't bad at all!

@elipsion
Copy link

elipsion commented Feb 8, 2021

I don't know enough about the innards of PS memory management to judge how feasible this is, but:

Just like bash and friends have $HISTSIZE, detailing how many previous commands to keep, couldn't all the problems regarding "But what if someone makes something very large?" be solved with a $LASTSIZE, specifying how many kB to keep in a $__/$last/$-. Shipping with a default of say 512kB shouldn't meaningfully impact the footprint of unsuspecting users, but will still keep the feature somewhat useful.

@mklement0
Copy link
Contributor

Good idea, @elipsion.

As an aside regarding the $PSDefaultParameterValues['Out-Default:OutVariable'] = '__' stopgap, because it has bitten me before: if you want to save the value of $__ after running a command, so that it survives subsequent commands, use $saved = $__.Clone() (or $saved = $($__)) - just $saved = $__ is not enough, as the ArrayList instance stored in $__ is reused.

@mattcargile
Copy link

As an aside regarding the $PSDefaultParameterValues['Out-Default:OutVariable'] = '__' stopgap, because it has bitten me before: if you want to save the value of $__ after running a command, so that it survives subsequent commands, use $saved = $__.Clone() (or $saved = $($__)) - just $saved = $__ is not enough, as the ArrayList instance stored in $__ is reused.

Thanks for sharing this. I was scratching my head why I couldn't get it to work when saving the output. I eventually used
write $__ -ov saved.
It seems to work in the same way as Clone()

Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

2 similar comments
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

@microsoft-github-policy-service microsoft-github-policy-service bot added Resolution-No Activity Issue has had no activity for 6 months or more labels Nov 16, 2023
Copy link
Contributor

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Discussion the issue may not have a clear classification yet. The issue may generate an RFC or may be reclassif Resolution-No Activity Issue has had no activity for 6 months or more WG-Engine core PowerShell engine, interpreter, and runtime
Projects
None yet
Development

No branches or pull requests