os/exec: Cmd.Dir is not checked when looking for executable on Windows

[atn832]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

(windows 10)
set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\atn\go
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

What did you do?

I created the following files in the folder of this source code file:
.\subfolder\list.bat: file content is simply "dir"
.\subfolder\list2.bat: file content is simply "dir"
.\list2.bat: file content is empty, or can be anything

package main

import (
	"fmt"
	"os"
	"os/exec"
)

func main() {
	// says it cannot find list.bat in the path.
	runScript("subfolder", "list.bat")
	// correctly runs .\subfolder\list2.bat from .\subfolder
	runScript("subfolder", "list2.bat")
	// runs .\subfolder\list.bat from .\
	runScript("", "subfolder\\list.bat")
	// runs .\subfolder\list2.bat from .\
	runScript("", "subfolder\\list2.bat")
}

func runScript(folder, filename string) bool {
	fmt.Println("Running", folder, filename)
	cmd := exec.Command(filename)
	cmd.Dir = folder
	// Stream to std out
	cmd.Stdout = os.Stdout
	cmd.Stdin = os.Stdin
	cmd.Stderr = os.Stderr

	err := cmd.Run()
	if err != nil && err.Error() != "exit status 1" {
		fmt.Println("Error running script: " + err.Error())
		return false
	}
	return true
}

Recipe:

• create the .bat files as described above
• build the executable from the go code above
• run the executable
• see that list.bat is not found, but list2.bat is indeed found

What did you expect to see?

I expect list.bat and list2.bat to list the content of subfolder.

What did you see instead?

list.bat is "not found", even though it is indeed in subfolder

EDIT: I've just go-fmt'd the source code as at https://play.golang.org/p/LDKmGQpxRH and also added the markdown tag for Go source code.

[odeke-em]

@atn832 thank you for the report!

However, your bug report has me a little confused:

        // says it cannot find list.bat in the path.
	runScript("subfolder", "list.bat")
	// correctly runs .\subfolder\list2.bat from .\subfolder
	runScript("subfolder", "list2.bat")

in that snippet adapted from your bug report, both list.bat and list2.bat are both children of subfolder yet one of them runs and the other doesn't run? That's funky, and would mean that if all on the outside is right, that Dir works only for list2.bat.

Also we have tests that set Dir that always run on every change, even on the Windows testing environments:

• go/src/os/exec/lp_windows_test.go

  Lines 65 to 95 in b7397c8

  	func (test lookPathTest) runProg(t *testing.T, env []string, args ...string) (string, error) {
  	cmd := Command(args[0], args[1:]...)
  	cmd.Env = env
  	cmd.Dir = test.rootDir
  	args[0] = filepath.Base(args[0])
  	cmdText := fmt.Sprintf("%q command", strings.Join(args, " "))
  	out, err := cmd.CombinedOutput()
  	if (err != nil) != test.fails {
  	if test.fails {
  	t.Fatalf("test=%+v: %s succeeded, but expected to fail", test, cmdText)
  	}
  	t.Fatalf("test=%+v: %s failed, but expected to succeed: %v - %v", test, cmdText, err, string(out))
  	}
  	if err != nil {
  	return "", fmt.Errorf("test=%+v: %s failed: %v - %v", test, cmdText, err, string(out))
  	}
  	// normalise program output
  	p := string(out)
  	// trim terminating \r and \n that batch file outputs
  	for len(p) > 0 && (p[len(p)-1] == '\n' || p[len(p)-1] == '\r') {
  	p = p[:len(p)-1]
  	}
  	if !filepath.IsAbs(p) {
  	return p, nil
  	}
  	if p[:len(test.rootDir)] != test.rootDir {
  	t.Fatalf("test=%+v: %s output is wrong: %q must have %q prefix", test, cmdText, p, test.rootDir)
  	}
  	return p[len(test.rootDir)+1:], nil
  	}

• go/src/os/exec/exec_test.go

  Lines 61 to 88 in b7397c8

  	func TestCommandRelativeName(t *testing.T) {
  	testenv.MustHaveExec(t)
  	// Run our own binary as a relative path
  	// (e.g. "_test/exec.test") our parent directory.
  	base := filepath.Base(os.Args[0]) // "exec.test"
  	dir := filepath.Dir(os.Args[0]) // "/tmp/go-buildNNNN/os/exec/_test"
  	if dir == "." {
  	t.Skip("skipping; running test at root somehow")
  	}
  	parentDir := filepath.Dir(dir) // "/tmp/go-buildNNNN/os/exec"
  	dirBase := filepath.Base(dir) // "_test"
  	if dirBase == "." {
  	t.Skipf("skipping; unexpected shallow dir of %q", dir)
  	}
  	cmd := exec.Command(filepath.Join(dirBase, base), "-test.run=TestHelperProcess", "--", "echo", "foo")
  	cmd.Dir = parentDir
  	cmd.Env = []string{"GO_WANT_HELPER_PROCESS=1"}
  	out, err := cmd.Output()
  	if err != nil {
  	t.Errorf("echo: %v", err)
  	}
  	if g, e := string(out), "foo\n"; g != e {
  	t.Errorf("echo: want %q, got %q", e, g)
  	}
  	}

  
  unless I am mistaken.

I don't have a Windows computer yet to reproduce this, I was just asking some questions.

[atn832]

The difference between list.bat and list2.bat is that i added a dummy list2.bat in the executable's folder (parent of subfolder), as I specified right under "What did you do?". So my file structure was like so:

.\test.exe
.\list2.bat (empty file)
.\subfolder\list.bat
.\subfolder\list2.bat

It seems that when test.exe checks to see if list2.bat exists, it will find one in test.exe's folder, so it will satisfy the check, then it will actually execute subfolder\list2.bat (as expected since cmd.Dir = "subfolder"). For list.bat, I guess it looks in the PATH environment variable, then in the executable's folder. Since it can't find it, it throws an error, even though it should really look for .\subfolder\list.bat.

It looks like the tests in lp_windows_test.go runs "cmd", which is in in one of the folders in the PATH environment variable. and exec_test.go uses the file's absolute path (exec.Command(filepath.Join(dirBase, base)...). So they don't test my use case.

[odeke-em]

Thanks for the response @atn832, I'll defer to @alexbrainman. I also don't have access to a Windows computer so couldn't dig through more.

[alexbrainman]

@atn832 I think this is working as expected.

list.bat is "not found", even though it is indeed in subfolder

This is a result of you running this line:

runScript("subfolder", "list.bat")

This sets Command's name parameter to "list.bat".

According to https://golang.org/pkg/os/exec/#Command

"... If name contains no path separators, Command uses LookPath to resolve name to a complete path if possible. Otherwise it uses name directly as Path. ..."

In your case it uses LookPath to search for "list.bat" (including current directory). It fails to find it, so you get that error message.

Alex

[AlexRouSg]

@atn832
I fixed your code for ya, read the comments explaining stuff.
https://play.golang.org/p/i_-R4uaU8W

[atn832]

Thank you all! I will try tonight.

I also tested on Linux and Mac, with a subfolder in cmd.Dir and simply "./somescript.sh" (without prepending the subfolder) and it worked fine. Maybe because it contains a separator?

[AlexRouSg]

@atn832
If your terminal is opened in ./subfolder then it will work that way. Go looks for the file (if you do not supply a full path) relative to the working directory which is set to the working directory of your terminal.

EDIT:
The working directory is set before running the command and "./" translates to current working directory. So yeah that's one way of doing it. But it would be safer to set the path in exec.Command since you can have a separate working directory.

[atn832]

Ok so I have tested both options + one other:

1. executing ".\list.bat" and ".\list2.bat" with cmd.Dir = "subfolder", and both scripts correctly listed the files in subfolder
2. executing "subfolder\list.bat" and "subfolder\list2.bat" by joining paths, with cmd.Dir = "subfolder", and it failed to find the script with the following message: Error running script: exec: "subfolder\\subfolder\\list.bat": file does not exist (same for list2.bat)
3. executing "subfolder\list.bat" (and 2), with cmd.Dir = "". As expected, it found and ran the script, but listed the files in the context of the executable's folder, not subfolder.

So I will use option 1, which is the same as what I used for Mac/Linux. Thank you!

[AlexRouSg]

@atn832
sry about the broken code, should've double checked things and actually ran it.
Actually there's a 4th option.
You can pass the absolute path to exec.Command(), I've actually ran this so I know it works.

As in the playground link, to get the path to where is the executable is.

ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := filepath.Dir(ex)

Then do something like
exec.Command(filepath.Join(exPath, folder, filename))