*: misc improvements to config command and substitute-path rules

A series of interconnected changes to both the terminal command
'config', DAP command 'dlv config', quality of life improvements to how
substitute-path works, and better documentation.

- Let 'config substitute-path' show the current substitute path rules
- Add a -clear command to 'config substitute-path'
- Support 'config-debug-info-directories'
- rewrite SubstitutePath to be platform independent (see below)
- document path substitution more

Regarding the rewrite of SubstitutePath: the previous version used
runtime.GOOS and filepath.IsAbs to determine which filepath separator to use
and if matching should be case insensitive. This is wrong in all situations
where the client and server run on different OSes, when examining core files
and when cross-compilation is involved.

The new version of SubstitutePath checks the rules and the input path to
determine if Windows is involved in the process, if it looks like it is it
switches to case-insensitive matching. It uses a lax version of
filepath.IsAbs to determine if a path is absolute and tries to avoid having
to select a path separator as much as possible

Fixes go-delve#2891, go-delve#2890, go-delve#2889, go-delve#3179, go-delve#3332, go-delve#3343

Documentation/cli/README.md

@@ -229,14 +229,24 @@ Changes the value of a configuration parameter.
 
 	config substitute-path <from> <to>
 	config substitute-path <from>
+	config substitute-path -clear
 
-Adds or removes a path substitution rule.
+Adds or removes a path substitution rule, if -clear is used all
+substitute-path rules are removed. Without arguments shows the current list
+of substitute-path rules.
+See also [Documentation/cli/substitutepath.md](//github.com/go-delve/delve/tree/master/Documentation/cli/substitutepath.md) for how the rules are applied.
 
 	config alias <command> <alias>
 	config alias <alias>
 
 Defines <alias> as an alias to <command> or removes an alias.
 
+	config debug-info-directories -add <path>
+	config debug-info-directories -rm <path>
+	config debug-info-directories -clear
+
+Adds, removes or clears debug-info-directories.
+
 
 ## continue
 Run until breakpoint or program termination.

Documentation/cli/starlark.md

@@ -29,6 +29,7 @@ raw_command(Name, ThreadID, GoroutineID, ReturnInfoLoadConfig, Expr, UnsafeCall)
 create_breakpoint(Breakpoint, LocExpr, SubstitutePathRules, Suspended) | Equivalent to API call [CreateBreakpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.CreateBreakpoint)
 create_ebpf_tracepoint(FunctionName) | Equivalent to API call [CreateEBPFTracepoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.CreateEBPFTracepoint)
 create_watchpoint(Scope, Expr, Type) | Equivalent to API call [CreateWatchpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.CreateWatchpoint)
+debug_info_directories(Set, List) | Equivalent to API call [DebugInfoDirectories](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.DebugInfoDirectories)
 detach(Kill) | Equivalent to API call [Detach](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Detach)
 disassemble(Scope, StartPC, EndPC, Flavour) | Equivalent to API call [Disassemble](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Disassemble)
 dump_cancel() | Equivalent to API call [DumpCancel](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.DumpCancel)

Documentation/cli/substitutepath.md

@@ -0,0 +1,66 @@
+## Path substitution configuration
+
+Normally Delve finds the path to the source code that was used to produce an executable by looking at the debug symbols of the executable.
+However, under [some circumstances](../faq.md#substpath), the paths that end up inside the executable will be different from the paths to the source code on the machine that is running the debugger. If that is the case Delve will need extra configuration to convert the paths stored inside the executable to paths in your local filesystem.
+
+This configuration is done by specifying a list of path substitution rules.
+
+
+### Where are path substitution rules specified
+
+#### Delve command line client
+
+The command line client reads the path substitution rules from Delve's YAML configuration file located at `$XDG_CONFIG_HOME/dlv/config.yml` or `.dlv/config.yml` inside the home directory on Windows.
+
+The `substitute-path` entry should look like this:
+
+```
+substitute-path:
+  - {from: "/compiler/machine/directory", to: "/debugger/machine/directory"}
+  - {from: "", to: "/mapping/for/relative/paths"}
+```
+
+If you are starting a headless instance of Delve and connecting to it through `dlv connect` the configuration file that is used is the one that runs `dlv connect`.
+
+The rules can also be modified while Delve is running by using the [config substitute-path command](./README.md#config):
+
+```
+(dlv) config substitute-path /from/path /to/path
+```
+
+Double quotes can be used to specify paths that contain spaces, or to specify empty paths:
+
+```
+(dlv) config substitute-path "/path containing spaces/" /path-without-spaces/
+(dlv) config substitute-path /make/this/path/relative ""
+```
+
+#### DAP server
+
+If you connect to Delve using the DAP protocol then the substitute path rules are specified using the substitutePath option in [launch.json](https://github.com/golang/vscode-go/blob/master/docs/debugging.md#launchjson-attributes).
+
+```
+	"substitutePath": [
+		{ "from": "/from/path", "to": "/to/path" }
+	]
+```
+
+The [debug console](https://github.com/golang/vscode-go/blob/master/docs/debugging.md#dlv-command-from-debug-console) can also be used to modify the path substitution list:
+
+```
+dlv config substitutePath /from/path /to/path
+```
+
+This command works similarly to the `config substitute-path` command described above.
+
+### How are path substitution rules applied
+
+Regardless of how they are specified the path substitution rules are an ordered list of `(from-path, to-path)` pairs. When Delve needs to convert a path P found inside the executable file into a path in the local filesystem it will scan through the list of rules looking for the first one where P starts with from-path and replace from-path with to-path.
+
+Empty paths in both from-path and to-path are special, they represent relative paths: 
+
+- `(from="" to="/home/user/project/src")` converts all relative paths in the executable to absolute paths in `/home/user/project/src`
+- `(from="/build/dir" to="")` converts all paths in the executable that start with `/build/dir` into relative paths.
+
+The path substitution code is SubstitutePath in pkg/locspec/locations.go.
+

Documentation/faq.md

@@ -116,6 +116,8 @@ The substitute-path feature can be used to solve this problem, see `help config`
 
 The `sources` command could also be useful in troubleshooting this problem, it shows the list of file paths that has been embedded by the compiler into the executable.
 
+For more informations on path substitution see [path substitution](cli/substitutepath.md).
+
 If you still think this is a bug in Delve and not a configuration problem, open an [issue](https://github.com/go-delve/delve/issues), filling the issue template and including the logs produced by delve with the options `--log --log-output=rpc,dap`.
 
 ### <a name="runtime"></a> Using Delve to debug the Go runtime

pkg/config/config.go

@@ -278,6 +278,7 @@ aliases:
 # between compilation and debugging.
 # Note that substitution rules will not be used for paths passed to "break" and "trace"
 # commands.
+# See also Documentation/cli/substitutepath.md.
 substitute-path:
   # - {from: path, to: path}
   

pkg/config/split.go

@@ -81,6 +81,9 @@ func ConfigureSetSimple(rest string, cfgname string, field reflect.Value) error
 			}
 			return reflect.ValueOf(&n), nil
 		case reflect.Bool:
+			if rest != "true" && rest != "false" {
+				return reflect.ValueOf(nil), fmt.Errorf("argument to %q must be true or false", cfgname)
+			}
 			v := rest == "true"
 			return reflect.ValueOf(&v), nil
 		case reflect.String:

pkg/locspec/locations.go

@@ -479,47 +479,99 @@ func (loc *NormalLocationSpec) findFuncCandidates(bi *proc.BinaryInfo, limit int
 	return r
 }
 
-func crossPlatformPath(path string) string {
-	if runtime.GOOS == "windows" {
-		return strings.ToLower(path)
+// isAbs returns true if path looks like an absolute path.
+func isAbs(path string) bool {
+	// Unix-like absolute path
+	if strings.HasPrefix(path, "/") {
+		return true
 	}
-	return path
+	return windowsAbsPath(path)
+}
+
+func windowsAbsPath(path string) bool {
+	// Windows UNC absolute path
+	if strings.HasPrefix(path, `\\`) {
+		return true
+	}
+	// DOS absolute paths
+	if len(path) < 3 || path[1] != ':' {
+		return false
+	}
+	return path[2] == '/' || path[2] == '\\'
+}
+
+func hasPathSeparatorSuffix(path string) bool {
+	return strings.HasSuffix(path, "/") || strings.HasSuffix(path, "\\")
+}
+
+func hasPathSeparatorPrefix(path string) bool {
+	return strings.HasPrefix(path, "/") || strings.HasPrefix(path, "\\")
 }
 
 // SubstitutePath applies the specified path substitution rules to path.
 func SubstitutePath(path string, rules [][2]string) string {
-	path = crossPlatformPath(path)
-	// On windows paths returned from headless server are as c:/dir/dir
-	// though os.PathSeparator is '\\'
-
-	separator := "/"                  // make it default
-	if strings.Contains(path, "\\") { // dependent on the path
-		separator = "\\"
+	// Look for evidence that we are dealing with windows somewhere, if we are use case-insensitive matching
+	caseInsensitive := windowsAbsPath(path)
+	if !caseInsensitive {
+		for i := range rules {
+			if windowsAbsPath(rules[i][0]) || windowsAbsPath(rules[i][1]) {
+				caseInsensitive = true
+				break
+			}
+		}
 	}
 	for _, r := range rules {
-		from := crossPlatformPath(r[0])
-		to := r[1]
+		from, to := r[0], r[1]
 
-		// If we have an exact match, use it directly.
+		// if we have an exact match, use it directly.
 		if path == from {
 			return to
 		}
 
-		// Otherwise check if it's a directory prefix.
-		if from != "" && !strings.HasSuffix(from, separator) {
-			from = from + separator
-		}
-		if to != "" && !strings.HasSuffix(to, separator) {
-			to = to + separator
+		match := false
+		var rest string
+		if from == "" {
+			match = !isAbs(path)
+			rest = path
+		} else {
+			if caseInsensitive {
+				match = strings.HasPrefix(strings.ToLower(path), strings.ToLower(from))
+				if match {
+					path = strings.ToLower(path)
+					from = strings.ToLower(from)
+				}
+			} else {
+				match = strings.HasPrefix(path, from)
+			}
+			if match {
+				// make sure the match ends on something that looks like a path separator boundary
+				rest = path[len(from):]
+				match = hasPathSeparatorSuffix(from) || hasPathSeparatorPrefix(rest)
+			}
 		}
 
-		// Expand relative paths with the specified prefix
-		if from == "" && !filepath.IsAbs(path) {
-			return strings.Replace(path, from, to, 1)
-		}
+		if match {
+			if to == "" {
+				// make sure we return a relative path, regardless of whether 'from' consumed a final / or not
+				if hasPathSeparatorPrefix(rest) {
+					return rest[1:]
+				}
+				return rest
+			}
 
-		if from != "" && strings.HasPrefix(path, from) {
-			return strings.Replace(path, from, to, 1)
+			toEndsWithSlash := hasPathSeparatorSuffix(to)
+			restStartsWithSlash := hasPathSeparatorPrefix(rest)
+
+			switch {
+			case toEndsWithSlash && restStartsWithSlash:
+				return to[:len(to)-1] + rest
+			case toEndsWithSlash && !restStartsWithSlash:
+				return to + rest
+			case !toEndsWithSlash && restStartsWithSlash:
+				return to + rest
+			case !toEndsWithSlash && !restStartsWithSlash:
+				return to + "/" + rest
+			}
 		}
 	}
 	return path

pkg/locspec/locations_test.go

@@ -1,7 +1,6 @@
 package locspec
 
 import (
-	"runtime"
 	"testing"
 )
 
@@ -69,16 +68,13 @@ func TestFunctionLocationParsing(t *testing.T) {
 }
 
 func assertSubstitutePathEqual(t *testing.T, expected string, substituted string) {
+	t.Helper()
 	if expected != substituted {
-		t.Fatalf("Expected substitutedPath to be %s got %s instead", expected, substituted)
+		t.Errorf("Expected substitutedPath to be %s got %s instead", expected, substituted)
 	}
 }
 
 func TestSubstitutePathUnix(t *testing.T) {
-	if runtime.GOOS == "windows" {
-		t.Skip("Skipping unix SubstitutePath test in windows")
-	}
-
 	// Relative paths mapping
 	assertSubstitutePathEqual(t, "/my/asb/folder/relative/path", SubstitutePath("relative/path", [][2]string{{"", "/my/asb/folder/"}}))
 	assertSubstitutePathEqual(t, "/already/abs/path", SubstitutePath("/already/abs/path", [][2]string{{"", "/my/asb/folder/"}}))
@@ -99,21 +95,17 @@ func TestSubstitutePathUnix(t *testing.T) {
 }
 
 func TestSubstitutePathWindows(t *testing.T) {
-	if runtime.GOOS != "windows" {
-		t.Skip("Skipping windows SubstitutePath test in unix")
-	}
-
 	// Relative paths mapping
 	assertSubstitutePathEqual(t, "c:\\my\\asb\\folder\\relative\\path", SubstitutePath("relative\\path", [][2]string{{"", "c:\\my\\asb\\folder\\"}}))
-	assertSubstitutePathEqual(t, "f:\\already\\abs\\path", SubstitutePath("F:\\already\\abs\\path", [][2]string{{"", "c:\\my\\asb\\folder\\"}}))
+	assertSubstitutePathEqual(t, "F:\\already\\abs\\path", SubstitutePath("F:\\already\\abs\\path", [][2]string{{"", "c:\\my\\asb\\folder\\"}}))
 	assertSubstitutePathEqual(t, "relative\\path", SubstitutePath("C:\\my\\asb\\folder\\relative\\path", [][2]string{{"c:\\my\\asb\\folder\\", ""}}))
-	assertSubstitutePathEqual(t, "f:\\another\\folder\\relative\\path", SubstitutePath("F:\\another\\folder\\relative\\path", [][2]string{{"c:\\my\\asb\\folder\\", ""}}))
+	assertSubstitutePathEqual(t, "F:\\another\\folder\\relative\\path", SubstitutePath("F:\\another\\folder\\relative\\path", [][2]string{{"c:\\my\\asb\\folder\\", ""}}))
 	assertSubstitutePathEqual(t, "my\\path", SubstitutePath("relative\\path\\my\\path", [][2]string{{"relative\\path", ""}}))
 	assertSubstitutePathEqual(t, "c:\\abs\\my\\path", SubstitutePath("c:\\abs\\my\\path", [][2]string{{"abs\\my", ""}}))
 
 	// Absolute paths mapping
 	assertSubstitutePathEqual(t, "c:\\new\\mapping\\path", SubstitutePath("D:\\original\\path", [][2]string{{"d:\\original", "c:\\new\\mapping"}}))
-	assertSubstitutePathEqual(t, "f:\\no\\change\\path", SubstitutePath("F:\\no\\change\\path", [][2]string{{"d:\\original", "c:\\new\\mapping"}}))
+	assertSubstitutePathEqual(t, "F:\\no\\change\\path", SubstitutePath("F:\\no\\change\\path", [][2]string{{"d:\\original", "c:\\new\\mapping"}}))
 	assertSubstitutePathEqual(t, "c:\\folder\\should_not_be_replaced\\path", SubstitutePath("c:\\folder\\should_not_be_replaced\\path", [][2]string{{"should_not_be_replaced", ""}}))
 
 	// Mix absolute and relative mapping

pkg/proc/bininfo.go

@@ -51,7 +51,7 @@ type BinaryInfo struct {
 	// GOOS operating system this binary is executing on.
 	GOOS string
 
-	debugInfoDirectories []string
+	DebugInfoDirectories []string
 
 	// BuildID of this binary.
 	BuildID string
@@ -676,7 +676,7 @@ func (bi *BinaryInfo) LoadBinaryInfo(path string, entryPoint uint64, debugInfoDi
 		bi.lastModified = fi.ModTime()
 	}
 
-	bi.debugInfoDirectories = debugInfoDirs
+	bi.DebugInfoDirectories = debugInfoDirs
 
 	return bi.AddImage(path, entryPoint)
 }
@@ -1396,7 +1396,7 @@ func loadBinaryInfoElf(bi *BinaryInfo, image *Image, path string, addr uint64, w
 	if err != nil {
 		var sepFile *os.File
 		var serr error
-		sepFile, dwarfFile, serr = bi.openSeparateDebugInfo(image, elfFile, bi.debugInfoDirectories)
+		sepFile, dwarfFile, serr = bi.openSeparateDebugInfo(image, elfFile, bi.DebugInfoDirectories)
 		if serr != nil {
 			return serr
 		}

pkg/terminal/command.go

@@ -502,13 +502,23 @@ Changes the value of a configuration parameter.
 
 	config substitute-path <from> <to>
 	config substitute-path <from>
+	config substitute-path -clear
 
-Adds or removes a path substitution rule.
+Adds or removes a path substitution rule, if -clear is used all
+substitute-path rules are removed. Without arguments shows the current list
+of substitute-path rules.
+See also Documentation/cli/substitutepath.md for how the rules are applied.
 
 	config alias <command> <alias>
 	config alias <alias>
 
-Defines <alias> as an alias to <command> or removes an alias.`},
+Defines <alias> as an alias to <command> or removes an alias.
+
+	config debug-info-directories -add <path>
+	config debug-info-directories -rm <path>
+	config debug-info-directories -clear
+
+Adds, removes or clears debug-info-directories.`},
 
 		{aliases: []string{"edit", "ed"}, cmdFn: edit, helpMsg: `Open where you are in $DELVE_EDITOR or $EDITOR