compatibility consideration for UseShellExecute with Process.Start()

[wfurt]

This is fresh start of topic related to dotnet/corefx#23705 and #23537

At least on Unix "UseShell" has standard meaning: shell is used to execute requested command.
There are many examples - like Python's popen() or simply difference between execv() and system().

On Windows his has different meaning and implies to open file with default application.
Unix implementation was recently updated to match Windows behavior.

However that possibly creates backward compatibility problem. 2.0 has been released with using /bin/sh to match traditional Unix experience.

Change in behavior can break existing usage of Process.Star() - and it did even in trivial case for corefx tests.
/bin/sh is pretty much guarantied to exist on any normal Unix system but new helpers may not.
Process.Start(new ProcessStartInfo { UseShellExecute = true, FileName ="ls"}) works with 2.0 but it does not work with current master branch on my system.

Further more, Unix users would expect normal shell behavior. Following examples will succeed when running with dotnet 2.0

        static void Main(string[] args)
        {
            String cmd="if [ -e /etc/passwd ] ; then echo yes ; else echo no; fi";
            var startInfo = new ProcessStartInfo(cmd);
            startInfo.UseShellExecute = true;
            startInfo.CreateNoWindow = true;
            using (Process px = Process.Start(startInfo))
            {
                    px.WaitForExit();
                    Console.WriteLine("cmd returned {0}", px.ExitCode);
            }
        }

One can pass in fragments of shell code with 2.0.

test@net-chip:~/process$ dotnet run
yes
cmd returned 0

another common use case is use if command pipeline and have shell handling it:

String cmd="cat /etc/passwd|grep root";

test@net-chip:~/process$ dotnet run
root:x:0:0:root:/root:/bin/bash
cmd returned 0

In this case shell will run each command in separate process and it will redirect output from cat to stdin of grep. This is pretty typical use.

Last part is use of .profile and shell variables. When used with shell, for example "echo $HOME" would expand to home directory. When used without shell, it would be passed in as string.

[danmoseley]

If a Unix oriented developer was writing code, she could well expect it to use sh as you say.
If a Windows developer was porting code, I suspect she would expect Process.Start(new ProcessStartInfo { UseShellExecute = true, FileName ="foo.txt"}); to continue to launch the installed editor as it does on Windows.

Which do we want to optimize for? And would we want another boolean to get the opposite?

We should double check whether we are currently following Mono behavior (our intent was we follow Mono). Of course we need not do so.

cc @marek-safar @stephentoub

[TSlivede]

I agree, that an option for launching sh -c might be useful, but if we need this, it should not be via UseShellExecute, because IMHO that's not what UseShellExecute is about.

_@stephentoub

because that's not what "UseShellExecute" is about

Says who?

https://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.useshellexecute.aspx :
The documentation speaks of shell not necessarily of command-line-shell:
_{see e.g. wiki for meanings of shell other than sh}

Gets or sets a value indicating whether to use the operating system shell to start the process

The documentation does however specify the behavieour:

When you use the operating system shell to start processes, you can start any document (which is any registered file type associated with an executable that has a default open action) and perform operations on the file, such as printing, by using the Process object. When UseShellExecute is false, you can start only executables by using the Process object.

---

Stackoverflow 1:

The longer answer is that the ShellExecute function is used to open a specified program or file - it is roughly equivalnt to typing the command to be executed into the run dialog and clicking OK, which means that it can be used to (for example):

• Open .html files or web using the default browser without needing to know what that browser is,
• Open a word document without needing to know what the installation path for Word is
• Run batch files
• Run any command on the PATH

---

Stackoverflow 2:

Without this set, you can only execute an EXE file directly. By setting this, you allow the Windows Shell to be used, which allows things such as specifying a .doc file and having the associated program open the file.
However, using the Windows Shell requires a valid desktop context, which is why your third use case fails.

---

Xamarin Documentation:

When you use the operating system shell to start processes, you can start any document (which is any registered file type associated with an executable that has a default open action) and perform operations on the file, such as printing, by using the Process object. When ProcessStartInfo.UseShellExecute is false, you can start only executables by using the Process object.

---

https://github.com/mairaw/docs/blob/master/xml/System.Diagnostics/Process.xml:

When the <xref:System.Diagnostics.ProcessStartInfo.UseShellExecute%2A?displayProperty=nameWithType> property is set to its default value, true, you can start applications and documents in a way that is similar to using the Run dialog box of the Windows Start menu. When <xref:System.Diagnostics.ProcessStartInfo.UseShellExecute%2A?displayProperty=nameWithType> is false, you can start only executables.

---

PS: Sorry for the huge size of this post.

[TSlivede]

Process.Start(new ProcessStartInfo { UseShellExecute = true, FileName ="ls"}) works with 2.0 but it does not work with current master branch on my system.

I agree that this should be fixed, because on Windows full .Net Framework (which I would consider to be the reference behaviour) UseShellExecute searches the Path-Variable and allows execution of executables (if Verb is not set or Verb="open").

[TSlivede]

another common use case is use if command pipeline and have shell handling it:

I'd suggest UseCommandLineShell for this behavior, see https://github.com/dotnet/corefx/issues/19956#issuecomment-326528130

[TSlivede]

windows.system.launcher, which is in https://github.com/dotnet/corefx/issues/20204 suggested for UseShellExecute for UWP, is also much more like the current behavior and not like /bin/sh -c

[stephentoub]

because IMHO that's not what UseShellExecute is about

That's all I was getting at, simply that it was an opinion rather than something documented.

[TSlivede]

The documentation I've linked doesn't explicitly forbid that UseShellExecute finally tries to open FileName via /bin/sh -c. However, it does document, that UseShellExecute allows direct opening of Documents - which /bin/sh -c doesn't allow, which is IMHO a strong argument against /bin/sh -c for UseShellExecute.

Additionally, using pipes (as in @wfurt's example) is also something that on Windows does not work via UseShellExecute, because UseShellExecute doesn't launch cmd.exe /c.

That said, if UseShellExecute in dotnet core on linux gets changed (for some reason I don't get) to sh -c, I think UseShellExecute in dotnet core on Windows should be changed to cmd.exe /c as well, to have some similar behavior. (But I'd definitly prefere some other option like UseCommandLineShell for this.)

[ghost]

I agree, that an option for launching sh -c might be useful, but if we need this, it should not be via >UseShellExecute, because IMHO that's not what UseShellExecute is about

Yeah - IMO, System.Diagnostics.Process should never have had a "UseShellExecute" even on Windows (let alone making it the default.) Why create an api whose very shape screams "This is for calling CreateProcessA/W" and then at the last minute, jump to a completely different api at a completely different layering level?

We can't go back and fix that blunder but let's not double down on it.

[marek-safar]

@danmosemsft I didn't check corefx behaviour but Mono tries to mimic .net Windows behaviour using /usr/bin/open on macos and xdg-open (+ few fallbacks) on Linux. Mono never directly calls /bin/sh as that's not portable.

In general, my recommendation would be try not to touch/improve this API but design a new one which is not Windows-centric and lives in a namespace which makes sense for new developers coming to the platform

[rmunn]

FWIW, dotnet/corefx@75f34a5 (which changes the behavior of UseShellExecute to use xdg-open on Linux instead of /bin/sh) has caused problems, because Linux developers see the name "UseShellExecute" and think it's going to use the shell. (There's only one thing called the "shell" in Linux, and that's /bin/sh and its variants, /bin/bash and so on). So when the logic of UseShellExecute was changed to NOT use the shell, it broke things like https://github.com/dotnet/templating/pull/1042/files.

Note in that dotnet-templating PR how the intent is to run the standard Linux chmod command with parameters specified by the user's script. The most common usage of this is in a post-action script like the following:

  "postActions": [
    {
      "condition": "(OS != \"Windows_NT\")",
      "description": "Make scripts executable",
      "manualInstructions": [{ "text": "Run 'chmod +x *.sh'" }],
      "actionId": "cb9a6cf3-4f5c-4860-b9d2-03a574959774",
      "args": {
        "+x": "*.sh"
      },
      "continueOnError": true
    }
  ]

This is supposed to result in the command chmod +x build.sh being run. What happens instead is:

Processing post-creation actions...
xdg-open: unexpected argument '+x'
Try 'xdg-open --help' for more information.
Unable to apply permissions +x to "*.sh".
Post action failed.
Description: Make scripts executable
Manual instructions: Run 'chmod +x *.sh'

This is happening with the released version of .Net Core 2.1.300, and it has broken a lot of templates that were relying on the chmod post-action from dotnet/templating#1042 to fix the dotnet/templating#1028 bugs. (The underlying cause of which is that NuGet doesn't understand the Unix extensions to .zip files; see dotnet/templating#1028 (comment) for extensive details).

The behavior of UseShellExecute not using the Linux shell was a major point of surprise for me, and by major point of surprise, I mean my thought was "Why in the world would you ever do it that way? That makes no sense!" I realize that cross-platform compatibility is important, but this really needs to be documented so that Linux developers will know that UseShellExecute doesn't actually mean what it says. And there needs to be some flag that actually means "Use /bin/sh", because Linux developers need that for a lot of scenarios.

[omajid]

See dotnet/core#1857 for a use-case broken by the change to Process.Start.

[danmoseley]

If I understand correctly, the main problem with existing behavior is that it broke code that worked on Unix on .NET Core 1.0-2.0 by changing to have different (but to some other people desirable, and consistent with Mono) behavior. Such code now needs to be written to explicitly find and launch the installed shell. And ideally we should have anticipated this break.

I agree with @marek-safar that hypothetically changing the behavior again would just cause more breaks and problems. It seems to me the way forward if we want to do anything is to add a way to invoke something with the actual shell as suggested by @TSlivede here - that is meaningful on both Windows and Unix.

I will close this, then, and if anyone is motivated, they can open such an API proposal.

cc @wtgodbe, @krwq as area owners feel free to reopen if you suggest otherwise.

[omajid]

I think there is still a compatibility issue here that needs addressing.

@TSlivede said:

I agree that this should be fixed, because on Windows full .Net Framework (which I would consider to be the reference behaviour) UseShellExecute searches the Path-Variable and allows execution of executables (if Verb is not set or Verb="open").

Consider this snippet on Linux:

using System;
using System.Diagnostics;

namespace CreateProcess
{
    class Program
    {
        static void Main(string[] args)
        {
            String cmd = args[0];
            var startInfo = new ProcessStartInfo(cmd);
            startInfo.UseShellExecute = true;
            startInfo.CreateNoWindow = true;
            using (Process px = Process.Start(startInfo))
            {
                    px.WaitForExit();
                    Console.WriteLine("cmd returned {0}", px.ExitCode);
            }
        }
    }
}

If we run it as:

$ dotnet --version
2.1.301
$ dotnet run dotnet
gio: file:///home/omajid/cliche/dotnet/CreateProcess/dotnet: Error when getting information for file “/home/omajid/cliche/dotnet/CreateProcess/dotnet”: No such file or directory
cmd returned 4
$ dotnet run /usr/bin/dotnet
gio: file:///usr/bin/dotnet: No application is registered as handling this file
cmd returned 4

So dotnet/corefx#23705 (or something else?) breaks executing of applications via $PATH and through a fully qualified path. Now UseShellExecute can no longer launch executibles (but can open documents). Both implementations are only partially compatible with what the API claims to do on full .NET Framework.

Edit: Ah, dotnet/corefx#24017 might be the culprit here. The original change dotnet/corefx#23705 used the mono approach of trying to execute a process directly before falling back to using xdg-open and friends.

[TSlivede]

@danmosemsft said:

I agree with @marek-safar that hypothetically changing the behavior again would just cause more breaks and problems.

I agree, that another incompatible change to this API should not be made.

However, at least "executing of applications via $PATH" could be fixed without breaking the current behaviour: If ProcessStartInfo.FileName doesn't contain any slashes (so it's not a path or URL) and there exists no file with the name ProcessStartInfo.FileName in the current folder, dotnet could try to search an executable with that name within the directories listed in PATH and execute that.

Sadly, I see no option, how "executing of applications [...] through a fully qualified path" could be fixed without breaking the current behaviour: A valid executable could still be opened in a reasonable way by xdg-open, gnome-open, etc. (e.g. open a shell script in an editor). A file that is not an executable could still be reasonably processed by exec-syscalls. (e.g. if the filetype is registered via binfmt_misc)

I think, that to support "executing of applications [...] through a fully qualified path" dotnet would need to

1. check if the file is an executable (has x-permission and is either a valid executable for the current kernel (ELF,...) or a script with shebang (#!) or registered via binfmt_misc (or something that I don't know))
2. if it's an executable: execute it
   else: use xdg-open, gnome-open, etc.

This would change the behaviour for script files (they are currently most likely opened in an editor and would then be executed. Therefore this is an incompatible change - however the new behaviour is closer to windows full .NET and does not convert a previously working call to this API to a call that produces an error, so maybe this should still be changed?

---

Anyway, all this is messy. I'd recommend to deprecate this API and provide a new clean one with explicit options, that only do exactly one thing each - not one flag that enables search in PATH AND open in default application AND open in new terminal...

As @marek-safar suggested, the naming of such an API shouldn't be windows-centric and should make sense to all developers. Maybe:

• OpenWithDefaultApplication for the current behaviour
• UseCommandLineShell for sh -c/cmd /c
• FindInPath to look for executables in the PATH environment variable
• OpenInTerminal to open in a new terminal

[danmoseley]

I'll let @wtgodbe, @krwq weigh in from herein out.

[tmds]

I like what @TSlivede is proposing.

However, at least "executing of applications via $PATH" could be fixed without breaking the current behaviour: If ProcessStartInfo.FileName doesn't contain any slashes (so it's not a path or URL) and there exists no file with the name ProcessStartInfo.FileName in the current folder, dotnet could try to search an executable with that name within the directories listed in PATH and execute that.

👍

we also need to detect file:// uri

check if the file is an executable (has x-permission and is either a valid executable for the current kernel (ELF,...) or a script with shebang (#!) or registered via binfmt_misc (or something that I don't know))

Maybe simplify this check to: regular file with execute bit set for user, group or other.

[rmunn]

One possible problem with the simple "execute bit set" check is that when common Windows filesystems (NTFS and FAT32) are mounted on Linux, they often end up mounted with 777 permissions: read, write and execute permissions for user, group, and other. This varies from distro to distro, but on the Linux Mint system I'm on right now (where I didn't change the default permission settings for mounting NTFS), I have an external hard disk with an NTFS partition, and its files are all showing up with read/write/execute bits set for all users.

So the simple "regular file with execute bit set" check may produce some false positives if the file is on an NTFS-mounted filesystem, and possibly in other circumstances as well. I think what @TSlivede proposed (executable bit set AND is a valid executable / a script with #! / etc.) is probably the right thing to check.

[tmds]

So the simple "regular file with execute bit set" check may produce some false positives if the file is on an NTFS-mounted filesystem, and possibly in other circumstances as well. I think what @TSlivede proposed (executable bit set AND is a valid executable / a script with #! / etc.) is probably the right thing to check.

Alternatively, instead of trying to figure out if it is a valid executable up front, we could fall back from execve returning ENOEXEC.

[tmds]

@wtgodbe, @krwq ptal at what is being proposed. This will improve consistency with how Windows behaves when setting UseShellExecute for executables/script files. If the proposal is ok, I can work on this.

[wtgodbe]

@tmds I think the gist of the current proposal makes sense, but is a little cloudy. Could you put together a formal API proposal (if, as @TSlivede suggests, you plan on deprecating the current API & replacing it)?

[tmds]

No API changes. When UseShellExecute is set, this would happen:

Current behavior:

use xdg-open, gnome-open, etc.

Proposed behavior:

if it's an executable: execute it
else: use xdg-open, gnome-open, etc.

[krwq]

@tmds @TSlivede I like some parts of the proposition (naming to be discussed) and I think we should not do too much magic in System.Process - it always leads to problems - I don't believe there is "one solution suits everyone" here. IMO UseShellExecute should be always doing 'sh -c' or something similar and not xdg-open/gnome-open/gvfs-open/gio open/kfmclient... (not sure if there is any customary/standardized name for this set of apps but will call it ExecutingApplications to simplify). I do not think we should do any pre-processing or analysis of what the user tries to do and just do what it asked to do with reasonable set of defaults.

Simple solution could be perhaps something similar to what @TSlivede has proposed:

string Shell { get; } = "sh -c {0}"; // or ExecutingApplication

and perhaps:

IEnumerable<string> ExecutingApplications { get; }

(this sounds more like a job for an external nuget package though which could also implement "DWIM" logic - personally not a fun of heuristics in the framework but got no problem with them as separate entities)

OpenInTerminal - IMO we should not have it - IMO it is separate problem
FindInPath - I think the OS/ExecutingApp should do this for you

[TSlivede]

@krwq As I already explained above, I beleave, that UseShellExecute must not call sh -c because that would mean, that on Windows and Linux there is an identical API-option but with completely different behavior (not in any way similar).

I can fully agree to this:

we should not do too much magic in System.Process

and could understand if the dotnet core team decides to remove all heuristics and xdg-open/gnome-open/gvfs-open/gio open/kfmclient... stuff - UseShellExecute can throw PlatformNotSupportedException, thats ok. But if sh -c is needed, then please use a different name than UseShellExecute.

[TSlivede]

OpenInTerminal - IMO we should not have it - IMO it is separate problem

I actually also don't really see the need for that feature.

I just included this suggestion, because it was requested here and because UseShellExecute on Windows actually has the additional side effect, of launching console applications
within a new conhost ^{(the terminal emulator on Windows)} instance.

[tmds]

Windows behavior for UseShellExecute is:

a. open:
- http://www.github.com -> open in browser
- myfile.xlsx           -> open in excel
- c:\                   -> open directory in explorer

b. execute files:
- script.vbs -> execute script
- winword    -> execute word

On Linux, by doing xdg-open/... it is possible to do a). We shouldn't remove this.
With the proposed change of detecting if FileName is an executable, it will also do b).

[krwq]

@tmds I agree that on Windows openning xlsx opens Excel - I do not say this is bad but as a counterargument please note that on Windows if you open command prompt and type foo.xlsx (on existing file) it will also open Excel while on Linux shell won't do it for you without explicitly using i.e. xdg-open.

One option perhaps could be adding something like: bool OpenNonExecutablesWithDefaultProgram

[tmds]

if you open command prompt and type foo.xlsx (on existing file) it will also open Excel while on Linux shell won't do it for you without explicitly using i.e. xdg-open.

Let's focus on behavior of the Process class.

One option perhaps could be adding something like: bool OpenNonExecutablesWithDefaultProgram

Even if UseShellExecute sounds to a Linux user as if it will use a shell program, we shouldn't make that the implementation. For compatibility/consistency reasons, we need to try to behave the same as Windows.

[TSlivede]

@krwq

on Windows if you open command prompt and type foo.xlsx (on existing file) it will also open Excel

Yes, but that doesn't mean UseShellExecute is using cmd.exe ("command prompt") or any other command-line-shell. There are things that work with UseShellExecute but not in cmd, and things that work with cmd but not with UseShellExecute.

UseShellExecute allows urls (like http://google.de) to be opened in your browser, cmd.exe does not allow this.
cmd allows pipes, input- and output-stream redirections, expansion of environment variables, etc. - none of these work with UseShellExecute.
And most importantly: UseShellExecute doesn't allow you to execute commands implemented within cmd.exe like dir or mklink.

[TSlivede]

Even if UseShellExecute sounds to a Linux user as if it will use a shell program,

UseShellExecute is executing a shell program: xdg-open/gnome-open/gvfs-open/gio open/kfmclient are shell programs, they are just not command-line-shell-programs. See wiki:

... the shell consists of an X window manager or [...], as well as of one or multiple programs providing the functionality to start installed applications ...

xdg-open/gnome-open/gvfs-open/gio open/kfmclient are "programs providing the functionality to start installed applications"

I know, that in the *NIX context "shell" basically always means command-line-shell. But AFAIK .NET was originally developed for Windows, so when the term shell is used in the API it almost certainly means shell as in Windows Shell - which is AFAIK closely related to the graphical userinterface of Windows and not to cmd.exe.

[krwq]

I think URLs and piping is something which convinces me more toward the proposed solution. I think it would be useful to compile a table of all scenarios we can think of which are expected to work on Windows and then from there figure out how to handle that on Linux so that we get maximum compat. It would be useful to also add stuff which is not working on Windows but currently does (or previously did) on Linux and decide what do we want to do with that going forward.

[TSlivede]

UseShellExecute on Windows allows using the parameter Verb. Some things that are possible with that parameter are documented here or here, some others here or here something similar to sudo. SHELLEXECUTEINFOW.lpVerb seems to be available as ProcessStartInfo.Verb.

However I don't think implementing all that within dotnet core is a good Idea...
I am increasingly convinced, that it would be best, to abandon the UseShellExecute API on *NIX and provide a clean alternative, that is available on Windows and *NIX.

[krwq]

@TSlivede I think abandoning completely won't work - we should cover at least most of the use cases (likely something really close to what you have proposed) and propose better alternative once we know what that is.

[TSlivede]

at least most of the use cases

If by this you mean "the most used use cases" - ok, this shouldn't be too hard. Partially reverting dotnet/corefx#24017 (try to execute directly before using xdg-open, etc) and implementing search in PATH and maybe implementing the verb runAs using sudo or similar should cover the most used use cases.

If by most cases you mean something like 80% of all different use cases - I don't think that's possible, because UseShellExecute allows using Verb - and there are many possible verbs because each application on Windows can register it's own verbs.

[TSlivede]

More strange things about UseShellExecute, that make it hard to emulate the exact behavior of UseShellExecute on Windows:

• UseShellExecute must be true for ProcessStartInfo.ErrorDialog
• WorkingDirectory behaves differently depending on whether UseShellExecute is true or false
• CreateNoWindow is ignoered if UseShellExecute is true

I think we should not try to emulate these strange things on *NIX...

[krwq]

@TSlivede I meant used use cases 😄 I'll see if we got some data about it available from nuget/github. I agree on what you just wrote (unless we got someone requesting any of that and has a valid scenario).

EDIT: unfortunately we only got usage counts but not details on how it was used...

[tmds]

I think we can close this one with the changes made in dotnet/corefx#33052.

[krwq]

Thank you a lot for fixing this @tmds!