Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get-Command reports non-executable files (documents) as applications #12625

Closed
mklement0 opened this issue May 11, 2020 · 34 comments
Closed

Get-Command reports non-executable files (documents) as applications #12625

mklement0 opened this issue May 11, 2020 · 34 comments
Labels
Issue-Question ideally support can be provided via other mechanisms, but sometimes folks do open an issue to get a Resolution-No Activity Issue has had no activity for 6 months or more WG-Cmdlets-Core cmdlets in the Microsoft.PowerShell.Core module

Comments

@mklement0
Copy link
Contributor

mklement0 commented May 11, 2020

See also: #12632, #12644, #12645

Note: Changing this would be a breaking change. That said, the current behavior is unhelpful, so the suggested change likely falls into Bucket 3: Unlikely Grey Area.

While you can pass a file path to Get-Command, any existing file - even if it isn't an executable (let's call it a document) - is currently reported as command of type Application.

The sensible behavior would be the following: Only return a file as a System.Management.Automation.ApplicationInfo instance:

  • on Unix-like platforms: if test -x for that file has an exit code of 0; that is, if the file is an executable from the OS' perspective and is executable by the current user.

  • on Windows: if the file's extension is listed in $env:PATHEXT

Otherwise, a non-terminating error should be reported, as is already done for non-existent commands.

It isn't useful to end users to report any file item as a command, let alone as a command of type Application, as is currently the case (use Get-Item to get information about non-executable files).

Conversely, reporting only true executables (files that the OS considers directly executable) is helpful on Unix-like platforms, given that executables typically have no filename extension at all, and even if they do it isn't the extension that determines executability, it is the permission bits.

In short: It is helpful to distinguish actual executables (from the OS' perspective) from documents that PowerShell - as a syntactic convenience - allows passing to an executable, implicitly (e.g., ./foo.txt being the same as Invoke-Item ./foo.txt). That PowerShell currently allows invoking documents by mere file name (foo.txt) via $env:PATH (only), is a related, but separate problem - see #12632.

Steps to reproduce

'hi' > t.txt
{ Get-Command -ea Stop ./t.txt } | Should -Throw

Expected behavior

The test should pass.

Actual behavior

The test fails:

Expected an exception, to be thrown, but no exception was thrown.

The reason is that the non-executable t.txt file was actually reported as type Application.

Environment data

PowerShell Core 7.1.0-preview.2
@mklement0 mklement0 added the Issue-Question ideally support can be provided via other mechanisms, but sometimes folks do open an issue to get a label May 11, 2020
@iSazonov iSazonov added the WG-Cmdlets-Core cmdlets in the Microsoft.PowerShell.Core module label May 12, 2020
@jhoneill
Copy link

It's an odd one. Profile in the example is a command of type "External Script" so that is ok.

Foo.xlsx is processable in Excel , and start Foo.xlsx has been valid for longer than we have had XLSx files. (I think start . worked in the Windows 95 version of command.com, but I don't trust my memory on the point.) Does working with start serve as a good proxy for _ is an application_ ? Typing
$path = foo.xlsx
When I mean
$path = 'foo.xlsx'
tells windows to start foo.xlsx and we take a pause while excel opens, so powershell is treating it as a command to run and the console output (none) goes into $path.

"If I can start it, it counts as an application" might be OK as a rule except that get-command C:\windows\system32\aadauthhelper.dll reports an application and you can't start a DLL.
Ideally there should be another command type for "launchable data file"

Technically a breaking change because someone might use get-command instead of test-path, but you could make it less so by returning "non-executable" if the input is a path and the path isn't something you can run.

@SeeminglyScience
Copy link
Collaborator

As @jhoneill said, that's definitely expected with ps1 files. Also even things that aren't directly executable can still be invoked with Invoke-Item (and that is indeed what & (gcm ./somefile.csv) does). It already doesn't turn up non-executable files with a wildcard search, if someone queries the full path that seems like they're pretty explicitly asking for this behavior.

@mklement0
Copy link
Contributor Author

Yes, sorry about the flawed $PROFILE example - I've corrected the OP to use a text file instead.

@mklement0
Copy link
Contributor Author

mklement0 commented May 12, 2020

Executable loses all meaning if it applies to any file: It is't the file itself that is executable in the case of ./t.txt; instead, precisely because it isn't executable, the system's GUI shell passes it to a true executable (typically for opening / viewing, not for carrying out an action).

The most sensible definition of command is: something that is itself executable and to which - at lest in principle - arguments can be passed.

if someone queries the full path that seems like they're pretty explicitly asking for this behavior.

While that may be true on Windows, it isn't on Unix platforms, and that is indeed what prompted creation of this issue (see this Stack Overflow question).

Notably, given that executables on Unix-like platforms typically have no filename extension at all, you cannot easily tell whether something is executable or not.
Also, even if it may be executable in principle, it may not be executable by you, due to lack of permissions.

The workaround is currently to use $(test -x ./t.txt; 0 -eq $LASTEXITCODE), but that is neither simple, nor obvious nor PowerShell-like.

Testing an individual file for executability also calls for adding an -Executable switch to Test-Path, which only returns $True for true executables.

# WISHFUL THINKING
# On Unix-like platforms: does ./foo have the executable permission bits set
# so that I can execute it?
Test-Path -Executable ./foo

However, doing so doesn't preclude also fixing Get-Command as proposed, especially if you consider the following use case:

# WISHFUL THINKING
# Get (only) all true executables in the current directory.
Get-Command ./*

@SeeminglyScience
Copy link
Collaborator

SeeminglyScience commented May 12, 2020

I like the idea of a separate cmdlet or parameter on something else to test executability.

I don't think there's anything to fix here though. It's doing exactly what it says on the tin, getting you a command that can be invoked. If the cmdlet was called Get-Executable I'd be right there with ya, but if it can make a command why wouldn't it?

@mklement0
Copy link
Contributor Author

mklement0 commented May 12, 2020

Understood and disagreed. I've given your comment the agreed-on thumbs-down, and conclude this aspect of the debate by reiterating: the current definition of command is meaningless, if any file is a command.

Distinguishing between a true executable, and a not-itself-executable document, on which a GUI shell operation can be performed, is useful.

Also note that ./txt is currently reported as type Application(!), and that there's no separate enum value for what "command type" ./txt represents (it would be something like Document, which, as the name suggests, makes it a non-command).

@SeeminglyScience
Copy link
Collaborator

Understood and disagreed. I've given your comment the agreed-on thumbs-down, and conclude this aspect of the debate by reiterating:

It does sort of lose the original spirit if you also provide a counter point though. I don't personally mind it however, I think it makes the conversation easier to navigate at a glance.

the current definition of command is meaningless, if any file is a command.

A command in this context is a PowerShell concept. It means something that PowerShell can invoke. It doesn't mean an executable file.

Distinguishing between a true executable, and a not-itself-executable document, on which a GUI shell operation can be performed, is useful.

Yeah for sure, that'd be a great command.

Also note that ./txt is currently reported as type Application(!)

Yeah I'd be a little bit more comfortable if it was of a CommandType called Item, but that's probably not worth the break or dev time.

@mklement0
Copy link
Contributor Author

It does sort of lose the original spirit if you also provide a counter point though.

Fair point: the comment started out as just a succinct reiteration of the original point - so as to clarify the gist of the disagreement - but then the spirit moved me to elaborate. And I'm still possessed:

A command in this context is a PowerShell concept.

It is, but it is a concept that is well-established historically, and I don't think that PowerShell in fundamental design intent deviates from it - as evidenced by the command-type enumeration members (System.Management.Automation.CommandTypes, and as evidenced that something like ./t.txt, which does not fit in there, is mistakenly reported as Application.

My sense is that if you ask the average PowerShell user: Is t.txt a command, they would tell you no, it's a document - and sensibly so.

Again: The distinction between (a) "a thing that is designed to carry out an action, typically operating on given input (pipeline, arguments)" and (b) "a piece of data stored in a file that can be acted on if passed to an (a)" is an intuitive and useful one.

Conversely: If you want the functionality that Get-Command currently exhibits - that is, if you don't care about this distinction - don't use Get-Command, use Get-Item.

Yeah I'd be a little bit more comfortable if it was of a CommandType called Item

To me that illustrates why such files should not be considered commands: essentially, by returning a type of Item what you're saying is: this is a command that isn't a command.

@SeeminglyScience
Copy link
Collaborator

Fair point: the comment started out as just a succinct reiteration of the original point - so as to clarify the gist of the disagreement - but then the spirit moved me to elaborate.

The whole thing is a trade off though, you give the other side the final word while still making it clear you disagree. If you want to give a thumbs down and still provide a counter point (or even just a summary), that's still fine obviously (imo), just not related to what we discussed.

It is, but it is a concept that is well-established historically, and I don't think that PowerShell in fundamental design intent deviates from it - as evidenced by the command-type enumeration members (System.Management.Automation.CommandTypes, and as evidenced that something like ./t.txt, which does not fit in there, is mistakenly reported as Application.

Well it's more of an implementation detail. It's processed by the NativeCommandProcessor (the command processor for ApplicationInfo objects), much in the same way a GUI application is processed. It'd be nice to have a cosmetic distinction, but it's not a mistake.

My sense is that if you ask the average PowerShell user: Is t.txt a command, they would tell you no, it's a document - and sensibly so.

Yeah sure, it's not a super well known feature.

Again: The distinction between (a) "a thing that is designed to carry out an action, typically operating on given input (pipeline, arguments)" and (b) "a piece of data stored in a file that can be acted on if passed to an (a)" is an intuitive and useful one.

It's going to act in mostly the same way as a GUI application.

Conversely: If you want the functionality that Get-Command currently exhibits - that is, if you don't care about this distinction - don't use Get-Command, use Get-Item.

If I'm using Get-Command it's because I want it as a CommandInfo object.

To me that illustrates why such files should not be considered commands: essentially, by returning a type of Item what you're saying is: this is a command that isn't a command.

Nope, just a different CommandType to signify the type of command it is.

@mklement0
Copy link
Contributor Author

The whole thing is a trade off though, you give the other side the final word while still making it clear you disagree. If you want to give a thumbs down and still provide a counter point (or even just a summary), that's still fine obviously (imo), just not related to what we discussed.

Point taken. I think we agree now that either use of thumbs-down can be useful.

Yeah sure, it's not a super well known feature.

Not only that: As I've just realized, on invocation of a document, it does something that is of questionable utility and likely not what the user intends: see #12632

If I'm using Get-Command it's because I want it as a CommandInfo object.

That will tell you nothing of interest about ./t.txt - especially not with respect to what true executable the document will be passed to.

@mklement0 mklement0 changed the title Get-Command reports non-executable files as applications Get-Command reports non-executable files (documents) as applications May 12, 2020
@SeeminglyScience
Copy link
Collaborator

Not only that: As I've just realized, on invocation of a document, it does something that is of questionable utility and likely not what the user intends: see #12632

It may not be what you personally expect, but it is by design. I use it every day.

If I'm using Get-Command it's because I want it as a CommandInfo object.

That will tell you nothing of interest about ./t.txt - especially not with respect to what true executable the document will be passed to.

It's not about information, CommandInfo is invokable with the invocation operators, FileSystemInfo isn't. It's an object that represents a command in more ways than formatting.

@mklement0
Copy link
Contributor Author

mklement0 commented May 12, 2020

I use it every day.

I'm genuinely curious: Please give me an example of when you open a document by mere file name (e.g, ReadMe.txt) that is located not in the current directory, but in (any) directory in $env:PATH, and how do you know remember which one will be opened, if multiple directories contain such a file?

It's not about information, CommandInfo is invokable with the invocation operators, FileSystemInfo isn't.

Fair point. That still doesn't make something that isn't a command but can be invoked as one a command, and supporting something like Get-Command .\* to find the true executables strikes me as far more useful than the ability to wrap a document in a CommandInfo.

@SeeminglyScience
Copy link
Collaborator

I use it every day.

I'm genuinely curious: Please give me an example of when you open a document by mere file name (e.g, ReadMe.txt) that is located not in the current directory, but in (any) directory in $env:PATH, and how do you know remember which one will be opened, if multiple directories contain such a file?

I don't, I do ./path/to/item.txt. But to answer your question, the same way you remember which exe will be invoked I suppose.

Fair point. That still doesn't make something that isn't a command but can be invoked as one a command

Remember that the word "command" has a very specific meaning in the context of PowerShell. What you are asking is for engine to no longer classify all items as commands. While I disagree, that's an understandable perspective to have. That said, your reasoning can't be that it isn't a command, because it already objectively is by design.

@mklement0
Copy link
Contributor Author

I do ./path/to/item.txt

So do I, and I wouldn't want to miss it - and I certainly didn't propose changing that.

To be clear: I was talking about invocation by mere file name - item.txt, in your example.

the same way you remember which exe will be invoked I suppose.

Multiple executables with a given file name in $env:PATH directories are the exception, multiple documents - such as README - aren't necessarily - which is why I think #12632 is problematic - no one expects to open documents via $env:PATH.

What you are asking is for engine to no longer classify all items as commands.

Indeed that's what I'm asking for, but that doesn't preclude sensible use of syntactic sugar such as ./path/to/item.txt:

That is, if an item turns out to be non-executable - a document - implicitly pass it to the Invoke-Item command, as a courtesy.

Incidentally, this could be extended to directory paths, so that, say, submitting / or . by itself would open the root or current folder in the GUI file manager.
Or, to put it differently: directories are items that are currently not considered commands by your conception, even though from the syntactic-sugar perspective it would be useful and provide consistent behavior.

With this conception:

@jhoneill
Copy link

Executable loses all meaning if it applies to any file: It is't the file itself that is executable in the case of ./t.txt; instead, precisely because it isn't executable, the system's GUI shell passes it to a true executable (typically for opening / viewing, not for carrying out an action).

Splitting hairs here, it doesn't say it is an executable. Long ago I was taught programs divided into utilities like format, sort, etc. and applications, word, multiplan, etc. OS programs were all utilities, never applications. And the application data files were a distinct class from configuration data. "App" has come to mean "program" and the distinction has been lost. PowerShell doesn't distinguish application program and application data file

It's not a bad assumption that "application" and "executable" are the same thing, but it doesn't hold here. Better naming would help.

@iSazonov
Copy link
Collaborator

I shortly look the code and it is not a side effect - the code explicitly does this (it seems excluding completor). So I'd said it is "by-design". If there are no security problems, I do not see the need to do the breaking change.

@mklement0
Copy link
Contributor Author

@iSazonov

Something being by design doesn't preclude changing it, if that design turns out to be unhelpful.

the code explicitly does this

The code explicitly does what?
Again, something like ./path/to/item.txt - by way of a path - opening item.txt in the default text editor (implicit Invoke-Item) is definitely welcome and mustn't change.

If there are no security problems

Invoking documents via $env:PATH is not a security problem per se, but dangerous in that you may end up operating on the wrong document; to use a real-world example from Windows 10:

PS C:\Users\jdoe> WindowsCodecsRaw.txt

The above opens C:\Windows\System32\WindowsCodecsRaw.txt - just because a file by that name happened to exist in a directory listed in $env:PATH.

Originally I mistakenly thought that a file by the same name in the current directory would take precedence, but that isn't true (I've also updated #12632) - the behavior is consistent in that not using a path (not using, e.g., ./WindowsCodecsRaw.txt) only looks in $env:PATH, as is the case with executables and scripts.

Still, I don't think it makes sense to locate and open documents this way. In the best-case scenario, it's an unhelpful feature that one won't use, in the worst-case scenario you'll mistakenly operate on the wrong file.

Neither does it make sense to report documents as Applications, or indeed to have Get-Command report them as commands at all.

In short: While a breaking change, I think the impact would be minimal (though I haven't try to analyze code out there):


@jhoneill

Splitting hairs here, it doesn't say it is an executable.

I should have said "command", which is a superset of "executable" (as a noun; as an adjective, it is the core quality of a command).

A simple way of conceptualizing the distinction I'm after: a command is a verb ("do something"), a document is a noun ("a thing").

Irrespective of what the original design intent was, observing this distinction is helpful, for the reasons outlined above.

The utility (CUI) vs. application (GUI) terminology has always been blurry on Windows, where the term "console application" is common, but I don't think that is relevant here.

Application data files are unequivocally what I've called documents above, distinct from the actual application executables, i.e., files containing executable code as recognized by the system.

Therefore, from the perspective of the command / document distinction, documents shouldn't be considered commands at all, and Get-Command shouldn't report them.

@vexx32
Copy link
Collaborator

vexx32 commented May 13, 2020

@mklement0 it seems to me the fundamental distinction here is simply one between your perceived definition of "command" and the one used in the design of the cmdlet.

If you're suggesting changes to the command, it's probably more productive to simply illustrate the precise differences you'd suggest and the practical benefits/losses and any breaking changes rather than getting into the semantics; that discussion is practically guaranteed to be unending.

The majority of the discussion here doesn't appear to come to any real resolution; the definition of "command" in this context does appear to be a bit looser than some definitions, but as others have pointed out... that is by design currently.

@mklement0
Copy link
Contributor Author

The OP always contained a concrete proposal, but I've now fleshed it out.

Similarly, I hope that the related, but distinct #12632 states its case clearly (and there should be no doubt that #12644 is a bug).

The attendant discussion was necessary to make the case for the proposal in the OP, and to suggest a conceptual reframing that I think will ultimately benefit users.

Again, that something is by design doesn't preclude changing it - if the impact is small enough, and the benefits are large enough.

@mklement0
Copy link
Contributor Author

I've spun out the Test-Path -Executable suggestion into #12645.

@yecril71pl
Copy link
Contributor

yecril71pl commented Aug 5, 2020

Incidentally, this could be extended to directory paths, so that, say, submitting / or . by itself would open the root or current folder in the GUI file manager.

Which, in case of a remote session, would be the immortal Norton Commander, I suppose 😛

Returning to indicative mode: when we encounter a number or a string, we do not execute any default action on it, we just emit it to the pipeline. If it ever comes back, the default action will be Write-Host. The same should happen with non-executable items we encounter. That is, . by itself should emit a DirectoryInfo, whereas t.txt should emit a FileInfo.

@mklement0
Copy link
Contributor Author

mklement0 commented Aug 5, 2020

@yecril71pl

when we encounter a number or a string, we do not execute any default action on it,

No: An unquoted string triggers argument-mode parsing, where the first argument is interpreted as a command and therefore executed.

This already happens for, say, ./file.txt (and for files that require quoting, you use & (& "./file 1.txt")).

For non-executable files, this is a convenient shortcut to Invoke-Item ./file.txt

There is no good reason not to offer the same convenience for directory items, so that executing C:\Windows (or
& "c:\program files"), for instance, would similarly act like Invoke-Item C:\Windows, which opens the folder in File Explorer (the host platform's GUI file manager).

Also, . by itself is currently invariably interpreted as the dot-sourcing operator, but in the absence of any further arguments I think falling back to Invoke-Item . would be perfectly reasonable.

(But keep in mind that all of this still doesn't justify reporting non-executable file-system items as type Application by Get-Command - this is a separate issue.)


How a terminal-only remote session should be dealing with such GUI shell operations is a valid question, but we already have that problem; e.g.:

# Currently FAILS QUIETLY - ditto with explicit `Invoke-Item c:\windows\system.ini`
icm -cn . { c:\windows\system.ini }

I've never considered this case before, but the quiet failure seems problematic.

@yecril71pl
Copy link
Contributor

When I write string, I mean string. When I write bareword, I mean bareword. And yes, I would prefer this convenient shortcut to cease and desist.

@mklement0
Copy link
Contributor Author

mklement0 commented Aug 5, 2020

I see. You should have been talking about barewords then, given that that's what you used in your examples:

That is, . by itself should emit a DirectoryInfo, whereas t.txt should emit a FileInfo.

Of course, if you truly meant quoted expression-mode strings, then the existing behavior mustn't change: '.' and 't.txt' must output verbatim . and t.txt, respectively.

Just to clarify: Are you advocating the removal of the existing ability to use, say, ./file.txt to open that file in the system's default GUI text editor (even if you're planning to have it do something different)?

If that is truly what you want, please open a new issue - though I don't ever see such a change happening (nor do I think there's a good reason for it).

@mklement0
Copy link
Contributor Author

Re-reading your original comment, I think I now see what you meant:

However, we have currently have an easy-to-grasp dichotomy:

  • A bareword as the first token that can't be interpreted as a number literal triggers argument-mode parsing where that bareword is invariably treated as a command to be invoked.

  • In case of a quoted string or a number-literal bareword, the value of that expression is output to the pipeline.

Leaving the breaking nature of your proposal with respect to bareword file paths aside:

What you're proposing - e.g. . emitting a DirectoryInfo instance to the pipeline - amounts to a potentially confusing blurring of this distinction.

It also doesn't naturally extend to file-system item paths that require quoting or are expressed via variables.

@yecril71pl
Copy link
Contributor

I envisage breaking rule #‌1, in that PowerShell should consider whether the bareword corresponds to a command or not.

  • If it is a command, execute it!
  • If it is an item that is not a command, emit its FileSystemInfo to the pipeline!
  • If it cannot be found, handle the error!

There are no file-system paths that require quoting, so I am not sure what you mean by that.

@mklement0
Copy link
Contributor Author

There are no file-system paths that require quoting, so I am not sure what you mean by that.

C:\Program Files, for instance. You cannot use that as a bareword, because it would be interpreted as executing C:\Program with argument Files.

@yecril71pl
Copy link
Contributor

C:\Program` Files\dotnet\dotnet.exe works like a charm.

@mklement0
Copy link
Contributor Author

Sure, you can work around the problem, but to be forced to do that (not being able to use single- or double-quoting) makes this an awkward solution. Also, it doesn't address variable-based paths; e.g., $env:ProgramFiles.

Aside from the conceptual problems with your proposal, let me restate that I think it has little chance of getting implemented for the reason that it'll break existing bareword non-executable file-path behavior alone. I suggest we stop discussing it here. If you really feel strongly enough to push for a breaking change, please open a new issue.

@yecril71pl
Copy link
Contributor

Anything you quote becomes a string in PowerShell. In order to have a bareword with funny characters, you need to escape them. Quoting is a work-around for people who find escaping too difficult.

Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

2 similar comments
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

@microsoft-github-policy-service microsoft-github-policy-service bot added Resolution-No Activity Issue has had no activity for 6 months or more labels Nov 16, 2023
Copy link
Contributor

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Question ideally support can be provided via other mechanisms, but sometimes folks do open an issue to get a Resolution-No Activity Issue has had no activity for 6 months or more WG-Cmdlets-Core cmdlets in the Microsoft.PowerShell.Core module
Projects
None yet
Development

No branches or pull requests

6 participants