Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support multiple exit codes based on what went wrong/right #1135

Merged
merged 24 commits into from
May 9, 2023
Merged

Conversation

CpuID
Copy link
Contributor

@CpuID CpuID commented Apr 23, 2023

Description, Motivation and Context

support multiple exit codes based on what went wrong/right

main motivator initially: failing helm preflights in an elegant way

0 = all passed, 3 = at least one failure, 4 = no failures but at least 1 warn

1 as a catch all, 2 for invalid input etc

resolves #1131

Checklist

  • New and existing tests pass locally with introduced changes.
  • Tests for the changes have been added (for bug fixes / features)
  • The commit message(s) are informative and highlight any breaking changes
  • Any documentation required has been added/updated. For changes to https://troubleshoot.sh/ create a PR here

Does this PR introduce a breaking change?

  • Yes
  • No

= at least one failure, 4 = no failures but at least 1 warn

1 as a catch all, 2 for invalid input etc

ref #1131
@CpuID CpuID added the type::feature New feature or request label Apr 23, 2023
@CpuID CpuID requested a review from a team as a code owner April 23, 2023 14:31
@CpuID CpuID self-assigned this Apr 23, 2023
@CpuID
Copy link
Contributor Author

CpuID commented Apr 23, 2023

TODO: docs

update: done replicatedhq/troubleshoot.sh#489

@CpuID
Copy link
Contributor Author

CpuID commented Apr 23, 2023

TODO: fix what I did with cli.RootCmd, to still propagate the exit code and allow all tests to pass

@CpuID
Copy link
Contributor Author

CpuID commented Apr 23, 2023

docs PR - replicatedhq/troubleshoot.sh#489

@CpuID
Copy link
Contributor Author

CpuID commented Apr 27, 2023

Ready for review 👍

Tests pass and docs are done replicatedhq/troubleshoot.sh#489

pkg/preflight/run.go Outdated Show resolved Hide resolved
Copy link
Member

@emosbaugh emosbaugh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like a more go-like pattern for exiting with different codes would be to define an interface

type ExitError interface {
	Error() string
	ExitStatus() int
}

func main() {
	err := RootCmd().Execute()
	fmt.Fprintln(os.Stderr, err.Error())
	switch err := errors.Unwrap(err).(type) {
	case ExitError:
		os.Exit(err.ExitStatus())
	default:
		os.Exit(1)
	}
}

@CpuID
Copy link
Contributor Author

CpuID commented May 4, 2023

I feel like a more go-like pattern for exiting with different codes would be to define an interface

I actually considered something like this, and even went to the extent of a custom error type (struct) that implements the error interface with extra methods and properties, but no matter how I tried, the RunE function in cobra insists on nothing else but a vanilla error type being returned which is rather annoying :( eg. like https://pkg.go.dev/errors#example-package

The idea of using errors.Unwrap() is a good one though... as an alternative to that delimiter approach, just to try and split up the exit code vs the error itself. It wasn't quite 100% what I was looking for though...

I did almost have some success again today with a custom error struct (to the point I got it to build), but it was still just a generic error being returned not my custom type.

If I change the RunE response param to be my custom type, I end up with something like this which is the biggest issue:

cmd/preflight/cli/root.go:85:9: cannot use func(cmd *cobra.Command, args []string) ExitCodeAndError {…} (value of type func(cmd *cobra.Command, args []string) ExitCodeAndError) as type func(cmd *cobra.Command, args []string) error in struct literal

I think for now in light of wanting to get this merged ASAP for use, we should retain the delimiter approach and as a future improvement try refactor things here.

@CpuID
Copy link
Contributor Author

CpuID commented May 4, 2023

@xavpaice also mentioned this example using Run instead of RunE - https://github.com/k8sgpt-ai/k8sgpt/blob/d8357ceb949e04d9dd21276a1d1dfcb60010c37a/cmd/analyze/analyze.go#L44

I've tried this approach also - the downside is it'll never run our PostRun hooks if we exit within Run, and we finish up our profiling logic in there right now... so I didn't bother going down this path (I'd like to keep PostRun as a usable thing)

@emosbaugh
Copy link
Member

Apologies for not testing the code in my suggestion. This was more difficult than I had thought. Ive created a quick prototype for returning errors with exit codes here.

Nathan Sullivan added 2 commits May 8, 2023 11:44
had to modify it slightly to work around nil pointer references on
errors, but it works :)
@CpuID
Copy link
Contributor Author

CpuID commented May 8, 2023

Apologies for not testing the code in my suggestion. This was more difficult than I had thought. Ive created a quick prototype for returning errors with exit codes here.

no worries thanks @emosbaugh :)

@CpuID CpuID requested a review from emosbaugh May 8, 2023 03:07
@CpuID CpuID requested a review from DexterYan May 8, 2023 03:07
cmd/preflight/cli/root.go Outdated Show resolved Hide resolved
cmd/preflight/cli/root.go Outdated Show resolved Hide resolved
Nathan Sullivan and others added 2 commits May 9, 2023 10:21
Co-authored-by: Ethan Mosbaugh <ethan@replicated.com>
Co-authored-by: Ethan Mosbaugh <ethan@replicated.com>
@CpuID CpuID requested a review from emosbaugh May 9, 2023 00:21
@CpuID
Copy link
Contributor Author

CpuID commented May 9, 2023

@emosbaugh the downside to always printing an error is we end up with the below, you get Error: with nothing after it at the end...

~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-invalid.yaml                                                                                         14:38:17
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- FAIL: nonexistent analyzer
      --- Analyzer not found
--- FAIL   preflight-sample
FAILED
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:03
3
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-pass.yaml                                                                                            10:23:05
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
--- PASS   preflight-sample
PASS
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:10
0
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml                                                                                            10:23:12
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
   --- FAIL: Must have at least 3 nodes in the cluster
      --- This application requires at least 3 nodes
--- FAIL   preflight-sample
FAILED
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:17
3

pkg/preflight/run.go Outdated Show resolved Hide resolved
@emosbaugh
Copy link
Member

@emosbaugh the downside to always printing an error is we end up with the below, you get Error: with nothing after it at the end...

~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-invalid.yaml                                                                                         14:38:17
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- FAIL: nonexistent analyzer
      --- Analyzer not found
--- FAIL   preflight-sample
FAILED
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:03
3
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-pass.yaml                                                                                            10:23:05
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
--- PASS   preflight-sample
PASS
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:10
0
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml                                                                                            10:23:12
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
   --- FAIL: Must have at least 3 nodes in the cluster
      --- This application requires at least 3 nodes
--- FAIL   preflight-sample
FAILED
Error: 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               10:23:17
3

I overlooked this logic on my previous review. All errors should have a message. You should set the error message to something like "preflight checks failed with errors" or "preflight checks failed with warnings". If you dont want to output a message in this case I would suggest another error type PreflightCheckResultsError and have another case in InitAndExecute with errors.As where you do not print the error.

if there's errors in processing (wasn't happening before), and we don't
return an error if all is well (for an exit code 0 situation)
@CpuID
Copy link
Contributor Author

CpuID commented May 9, 2023

OK I feel like this might be the best we're going to get... keen to get this merged and then improve on it later if we need to though :)

~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-pass.yaml                                                                                 
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
--- PASS   preflight-sample
PASS
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               
0
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml                                                                                            
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
   --- FAIL: Must have at least 3 nodes in the cluster
      --- This application requires at least 3 nodes
--- FAIL   preflight-sample
FAILED
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               
3
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-invalid.yaml                                                                                         
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   

   --- FAIL: nonexistent analyzer
      --- Analyzer not found
--- FAIL   preflight-sample
FAILED
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               
3
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ # ^^ this ones hard to return a 2 for (instead of a 3), as its in the analyzer code itself. I didn't want to end up with a bunch of extra scope creep, left it as a 3
quote> 
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-pass.yaml --format asdf                                                                        
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   
Error: unknown output format: "asdf"
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ echo $?                                                                                                                                               
2

@CpuID CpuID requested a review from emosbaugh May 9, 2023 06:55
cmd/preflight/cli/root.go Outdated Show resolved Hide resolved
@CpuID CpuID requested a review from emosbaugh May 9, 2023 21:09
@CpuID
Copy link
Contributor Author

CpuID commented May 9, 2023

~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml --format json                                                                              07:50:40
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   
{
  "pass": [
    {
      "title": "Required Kubernetes Version",
      "message": "Your cluster meets the recommended and required versions of Kubernetes."
    }
  ],
  "fail": [
    {
      "title": "Must have at least 3 nodes in the cluster",
      "message": "This application requires at least 3 nodes"
    }
  ]
}

@CpuID
Copy link
Contributor Author

CpuID commented May 9, 2023

~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml >/dev/null                                                                                 07:51:38
name: cluster-resources    status: running         completed: 0    total: 2   
name: cluster-resources    status: completed       completed: 1    total: 2   
name: cluster-info         status: running         completed: 1    total: 2   
name: cluster-info         status: completed       completed: 2    total: 2   
~/git/replicatedhq/troubleshoot cpuid-1131 *6 ?3 ❯ bin/preflight --interactive=false 1131-preflight-fail.yaml 2>/dev/null                                                                                07:51:43

   --- PASS Required Kubernetes Version
      --- Your cluster meets the recommended and required versions of Kubernetes.
   --- FAIL: Must have at least 3 nodes in the cluster
      --- This application requires at least 3 nodes
--- FAIL   preflight-sample
FAILED

pkg/preflight/run.go Outdated Show resolved Hide resolved
pkg/preflight/run.go Outdated Show resolved Hide resolved
pkg/preflight/run.go Outdated Show resolved Hide resolved
Nathan Sullivan and others added 4 commits May 10, 2023 08:04
Co-authored-by: Ethan Mosbaugh <ethan@replicated.com>
Co-authored-by: Ethan Mosbaugh <ethan@replicated.com>
Co-authored-by: Ethan Mosbaugh <ethan@replicated.com>
@CpuID CpuID merged commit 3548b46 into main May 9, 2023
21 checks passed
@CpuID CpuID deleted the cpuid-1131 branch May 9, 2023 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type::feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

When preflight results in FAILED, it exits with code 0
4 participants