"Better" support for adding progress to long-running commands? #14710
jimbobmcgee
started this conversation in
General
Replies: 1 comment
-
It is possible to embed the However, if you want to calculate the actual percentage, it is required that you catch all the pipeline input in the Note also that not all pipelines are finite sequences (for example, an event stream) or will only abort after an external signal (break, timed event etc..). Reading all pipeline input ahead of time may also cause weird effects in certain scripts. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I recently ran a one-liner against a file that was much bigger than the ones I usually ran it against and, when that (obviously) took a lot longer to finish than I hoped, thought "I probably wouldn't mind so much, if this had a progress bar".
That turned out to be just annoying enough to implement, that I wondered, "surely there's a better way...?" I thought I'd start this discussion to see if we could come up with one...
Consider the following, pretty-standard, "objectify a log file and then do something with it" code. The log line format is unimportant for the discussion, but it's basically
time,msgtype,sessionid,messagetext
– assume the regex is suitable:My first thought was "put a
Write-Progress
at the top of theForEach-Object
. That's OK but, now, how to determine the-PercentageComplete
? Without it, you're still no clearer on how long the above is going to take. It's pretty basic maths –100 * position / length
– but you have no access to reasonableposition
orlength
values.Select-String
will yield the line number (position
) but gettingLineNumber
means giving up on-ExpandProperty Matches
and introducing a secondforeach
somewhere, so you can join upLineNumber
andMatch
....and then, unless you scan the file twice, you don't have a total line count (
length
), so you can't get any further with that.Maybe introduce
Get-Item
? That would get you access to a total byte length, without having to scan the file, and theMatch.Index
property is supposed to give the position of the match...At first glance this seems OK...but no. Seems
Select-String
goes line-by-line (as it should), and so$_.Index
is relative to the line, not the overall input. Since the Pattern is a^...$
expression, the match is always the entire line, and$_.Index
is always going to be 0. Besides,$_.Index
is a character position, and$file.Length
is a byte length, and one day encoding is going to come along and make those not play nicely together.In then end, had to forego the idiomatic approach and settle on...
...and, while that isn't a drastic change from the original, it's still a shift from the more-idiomatic pipelined statement. A larger command or procedure might suffer:
$sr
and$line
variables I hope I'm not clobbering from somewhere (I could$private:
-scope these, I suppose)while
loop instead of the idiomaticForEach-Object
Reporting progress should be easy, and now I'm...sad...for the inelegance I've had to introduce. What started as a nice, pasteable snippet now has to be a function or script.
So, can anything be done to improve the progress-reporting experience in an idiomatic fashion?
I thought of a few ideas, but these come wholly from the perspective of a PS user, not a PS developer, and I have little input much effort something like this would take. Even as a user, I've pretty much talked myself out of all of them:
Have certain cmdlets emit a percentage property of their own?
Would be relatively easy to drop into an existing script:
But that might work for
Select-String
, but what about other cmdlets?Get-Content
would be an obvious candidate, but that doesn't return a structured object, and you don't want to break existing scripts by changing the output of something as fundamental asGet-Content
.Ratify a known variable, such as
$PSProgress
, which cmdlets can choose to update?Requires support from cmdlets, but seems more light-touch to my eye. Cmdlets that take a
-Path
argument could update$PSProgress
if they know their length and position in advance; cmdlets that don't (or where input is piped) do not. Somewhere along the line, something might have updated$PSProgress
, and we can get some visual feedback from that:That said, introducing new variables would affect anyone who had inadvertantly defined the same name already (variables named
$PSSomething
are not restricted, after all), and the global nature of the variable may lead one cmdlet to clobber the value for another – I could seeGet-ChildItem
setting one up for the files in a directory, thenGet-Content
immediately overwriting it, for example.Ratify a
-ShowProgress
switch parameter, and introduce in certain cmdlets?"Solves" the
Get-Content
problem by not introducing an output property. "Solves" the clobbering problem by not relying on global state. Easy enough for a user to choose what shows progress and what doesn't:But it's a lot of cmdlets to go through to add in support, and clutters up the implementation of those cmdlets with progress-reporting stuff. In general, this is the thing I end up adding to my own cmdlets, and it's always a pain to have to do.
Have
Write-Progress
take a pipeline input and emit the same object as output, and attempt to identify progress from that?I almost like this one, as it appears to localise the problem to
Write-Progress
, and again is easy enough for a user to invoke, by inserting into the pipeline where you want it. Shouldn't have backward-compatibility issues, asWrite-Progress
doesn't currently take input or yield output:But I can't see how the implementation could be anything other than magic, and that probably isn't desirable. Stuff like
if ($_ -is [IO.Stream]) { ... } elseif ($_ -is [ICollection]) { ... } ...
would clutter the implementation ofWrite-Progress
, and would probably need as much upstream support from individual cmdlets as the others, arguably needing something like (1) or (2) anyway, for the common case.So I open it up to the community. I'm sure there are other ideas that I haven't considered, and would love to understand your take on them...
Beta Was this translation helpful? Give feedback.
All reactions