"Shell exec failed" on 0.26.4 to 0.26.6 Windows OS

[ehfd]

Describe the bug
A clear and concise description of what the bug is.

Shell does not start on version 0.26.4 to 0.26.6 on Windows from a pod.

To Reproduce
Steps to reproduce the behavior:

1. On Windows, use K9s 0.26.4 to 0.26.6
2. Use <s> on a pod to start a shell
3. See error "Shell exec failed"

Expected behavior
A clear and concise description of what you expected to happen.

Shell should be started. It fails with "Shell exec failed".

Versions (please complete the following information):

• OS: Windows
• K9s: 0.26.4 to 0.26.6, WORKS on 0.26.3 and kubectl exec -it pod -- /bin/bash
• K8s: 1.22.12

Additional context
Add any other context about the problem here.

Works on 0.26.3. Does not work on 0.26.4 to 0.26.6.

[derailed]

@ehfd Can you include k9s debug logs? It should print out the command it's trying to exec. Once you see that command cp/paste in your shell and see what it's complaining about. There was a fix with exec for linux in 0.26.4, which could have hosed windows??

[ehfd]

@derailed Before I do that, can I understand that as "you could not reproduce it yourself"?

[ehfd]

�[90m11:37PM�[0m �[32mINF�[0m 🐶 K9s starting up...
�[90m11:37PM�[0m �[31mWRN�[0m Unable to locate K9s config. Generating new configuration...
�[90m11:37PM�[0m �[32mINF�[0m ✅ Kubernetes connectivity
�[90m11:37PM�[0m �[31mWRN�[0m No context specific skin file found -- C:\Users\User\AppData\Local\k9s\nautilus_skin.yml
�[90m11:37PM�[0m �[31mWRN�[0m No skin file found -- C:\Users\User\AppData\Local\k9s\skin.yml. Loading stock skins.
�[90m11:37PM�[0m �[31mWRN�[0m Custom view load failed C:\Users\User\AppData\Local\k9s\views.yml �[36merror=�[0m�[31m"open C:\\Users\\User\\AppData\\Local\\k9s\\views.yml: The system cannot find the file specified."�[0m
�[90m11:37PM�[0m �[31mWRN�[0m CustomView watcher failed �[36merror=�[0m�[31m"GetFileAttributes: The system cannot find the file specified."�[0m
�[90m11:37PM�[0m �[31mWRN�[0m os detect failed �[36merror=�[0m�[31m"no os information available"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m Shell exec failed
�[90m11:37PM�[0m �[31mWRN�[0m os detect failed �[36merror=�[0m�[31m"no os information available"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m Shell exec failed
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m Edit exec failed
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m Edit exec failed
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m11:37PM�[0m �[1m�[31mERR�[0m�[0m Edit exec failed

Ahh, I think I might understand that. kubectl is not in my path. But why does it work until 0.26.3 and not from 0.26.4?
It works in 0.26.3 without kubectl needed in the path.

[slimus]

I guess because we updated k8s libs?

[ehfd]

@slimus Is K9s supposed to have kubectl externally for it to run properly then?

[slimus]

@ehfd yes. kubectl should be presented in your system and available from your PATH.

[ehfd]

Then, the difference until 0.26.3 was that everything worked when I just put kubectl in the same directory as k9s. Without PATH.

[slimus]

I'm not sure, but maybe this is related to your issue https://go.dev/src/os/exec/exec.go

// The functions Command and LookPath look for a program
// in the directories listed in the current path, following the
// conventions of the host operating system.
// Operating systems have for decades included the current
// directory in this search, sometimes implicitly and sometimes
// configured explicitly that way by default.
// Modern practice is that including the current directory
// is usually unexpected and often leads to security problems.

// To avoid those security problems, as of Go 1.19, this package will not resolve a program
// using an implicit or explicit path entry relative to the current directory.
// That is, if you run exec.LookPath("go"), it will not successfully return
// ./go on Unix nor .\go.exe on Windows, no matter how the path is configured.
// Instead, if the usual path algorithms would result in that answer,
// these functions return an error err satisfying errors.Is(err, ErrDot).

[ehfd]

Well, unfortunate then. Probably a recent change.

[ehfd]

// These will not find and run ./prog or .\prog.exe,
// no matter how the current path is configured.
//
// Code that always wants to run a program from the current directory
// can be rewritten to say "./prog" instead of "prog".
//
// Code that insists on including results from relative path entries
// can instead override the error using an errors.Is check:
//
//	path, err := exec.LookPath("prog")
//	if errors.Is(err, exec.ErrDot) {
//		err = nil
//	}
//	if err != nil {
//		log.Fatal(err)
//	}
//	use(path)
//
// and
//
//	cmd := exec.Command("prog")
//	if errors.Is(cmd.Err, exec.ErrDot) {
//		cmd.Err = nil
//	}
//	if err := cmd.Run(); err != nil {
//		log.Fatal(err)
//	}
//
// Setting the environment variable GODEBUG=execerrdot=0
// disables generation of ErrDot entirely, temporarily restoring the pre-Go 1.19
// behavior for programs that are unable to apply more targeted fixes.
// A future version of Go may remove support for this variable.
//
// Before adding such overrides, make sure you understand the
// security implications of doing so.
// See https://go.dev/blog/path-security for more information.

Right below that line, it does show how it can detect executables in the current directory, though.
And I guess k9s will have no harm changing the error behavior to this.

[slimus]

@ehfd it would be nice if you can create PR. Thanks!

[ehfd]

Will consider it. Could you point where this relevant code is tho?

[slimus]

For sure:

k9s/internal/view/exec.go

Line 42 in c66002d

func runK(a *App, opts shellOpts) bool {

[ehfd]

PR #1802

[ehfd]

This is what @slimus wrote in #1802:

@derailed I was thinking about this problem for the last couple of days. And I have an idea of how to fix it and don't have any global security issues. We can add a new config execKubectl: kubectl (default value) and use it in our code. @ehfd and others can change it to execKubectl: ./kubectl. And also with this change, I can use OpenShift cli execKubectl: oc instead of kubectl, because OpenShift cluster has its own cli tool. What do you think about it? Thanks a lot.

Rather than this, what about adding a command line option like --localkubectl? I tend to just remove the %LocalAppData%\k9s folder when I'm finished in portable purposes. So basically, if that command line option is invoked, the code block I wrote in #1802 is conditionally activated. And for OpenShift, we can add another command line option --execkubectl for changing the name of the executable.

[derailed]

@slimus Thank you Alex!! I thought this made perfect sense too! We should try a PR for it, as @ehfd PR PR would not have worked unless the cmd is specified as ./kubetcl or add . to the PATH. The quick resolve here imho it to use something like direnv to set the path to your kubectl of choice or alias kubectl as oc. This feel pretty edge case to me as most respectable k8s cluster admins would setup kubectl and k9s in their shell and be done already. On the other hand if one want to use k8s envs in a temporal fashion, then perhaps the container route is advisable.
That said, I am definitely not keen to introduce 2 CLI args to fit that bill...

[AntonioND]

I'm having this issue with version v0.27.2, except I have the kubectl binary in my PATH (installed by the Kubernetes plugin for VS Code) C:\Users\<username>\.azure-kubectl\kubectl.exe.

If I close K9s I can invoke kubectl as usual without specifying the full path, but K9s doesn't find it.

[obamor]

Hello, I have the same Issue as @AntonioND

Windown 11
k9s: v0.27.2
kubectl: v1.26.1 (C:\ProgramData\chocolatey\bin\kubectl.exe)

kubectl is available via PATH

Logs of k9s

�[90m6:27PM�[0m �[1m�[31mERR�[0m�[0m kubectl command is not in your path �[36merror=�[0m�[31m"exec: \"kubectl\": cannot run executable found relative to current directory"�[0m
�[90m6:27PM�[0m �[1m�[31mERR�[0m�[0m Edit exec failed

[AntonioND]

I've modified k9s to print the PATH right before attempting to run kubectl, and the list of directories includes the one where I have kubectl.exe. I've decided to do a bit of digging, and I think it's a problem in how the Go library behaves when it detects the binary in the current working directory.

This is LookPath():
https://cs.opensource.google/go/go/+/fd208c8850cdfe84469d3e4ef9bd55e2e0cece70:src/os/exec/lp_windows.go;l=90-129

And this is the documentation of DotError:
https://cs.opensource.google/go/go/+/fd208c8850cdfe84469d3e4ef9bd55e2e0cece70:src/os/exec/exec.go;l=1296-1303

I've just realized I had a kubectl.exe in the same directory as my k9s.exe. LookPath() detects it and ignores everything else I have in my PATH (where I have kubectl.exe in not one, but two locations!). After removing the kubectl.exe of my working directory, it can find it in my PATH.

EDIT: I also tried to set GODEBUG=execerrdot=0 but it didn't work for some reason https://cs.opensource.google/go/go/+/fd208c8850cdfe84469d3e4ef9bd55e2e0cece70:src/os/exec/exec.go;l=83-86

[AntonioND]

I've created a PR to detect failures due to my problem and write the reason to the log.

[ehfd]

I still think that there should be a command-line option and/or k9s configuration to allow this.

[slimus]

@ehfd let's create a new issue with improvements and close this one. What do you think?

[ehfd]

@slimus at your convenience.

[PapaySail]

scoop install kubectl is a solution for me.

[sasaxena-equinix]

scoop install kubectl is a solution for me.

Thanks @PapaySail, this works for me too.

[ehfd]

For Windows users, add $env:Path = "$env:Path;$PWD" if you are using Powershell and set "PATH=%PATH%;%CD%" when using cmd or a .bat file (in this case, use %~dp0 over %CD%).