Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In some cases okteto deploy is not returning the same output than the command executed #2663

Closed
ifbyol opened this issue May 9, 2022 · 5 comments · Fixed by #2897
Closed
Assignees
Labels
bug Something isn't working customer reported

Comments

@ifbyol
Copy link
Member

ifbyol commented May 9, 2022

Describe the bug
A customer has reported that the okteto deploy command was returning a cryptic error deploying a helm application

!  client.go:128: [debug] creating 20 resource(s)
 !  Error: an error on the server ("") has prevented the request from succeeding (post deployments.apps)
 !  helm.go:84: [debug] an error on the server ("") has prevented the request from succeeding (post deployments.apps)

If the same command is executed directly from the terminal, the output is different, you can see the real error there:

upgrade.go:431: [debug] warning: Upgrade "ld" failed: failed to create resource: Deployment in version "v1" cannot be handled as a Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.Value: ReadString: expects " or n, but found t, error found in #10 byte of ...|,"value":true}
Error: UPGRADE FAILED: failed to create resource: Deployment in version "v1" cannot be handled as a Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.Value: ReadString: expects " or n, but found t, error found in #10 byte of ...|,"value":true}
helm.go:84: [debug] Deployment in version "v1" cannot be handled as a Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.Value: ReadString: expects " or n, but found t, error found in #10 byte of ...|,"value":true}
failed to create resource
helm.sh/helm/v3/pkg/kube.(*Client).Update.func1
	helm.sh/helm/v3/pkg/kube/client.go:235
helm.sh/helm/v3/pkg/kube.ResourceList.Visit
	helm.sh/helm/v3/pkg/kube/resource.go:32
helm.sh/helm/v3/pkg/kube.(*Client).Update
	helm.sh/helm/v3/pkg/kube/client.go:219
helm.sh/helm/v3/pkg/action.(*Upgrade).releasingUpgrade
	helm.sh/helm/v3/pkg/action/upgrade.go:376
runtime.goexit
	runtime/asm_amd64.s:1581
UPGRADE FAILED
main.newUpgradeCmd.func2
	helm.sh/helm/v3/cmd/helm/upgrade.go:199
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.3.0/command.go:902
main.main
	helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	runtime/proc.go:255
runtime.goexit
	runtime/asm_amd64.s:1581

This makes really hard to debug the issue just checking the logs returned by Okteto.

Maybe the proxy is not handling right the error

To Reproduce
Steps to reproduce the behavior:

I created a branch in a repository with a way to reproduce the issue. Repository https://github.com/ifbyol/movies and branch ifbyol/invalid-chart.

  1. Deploy from the UI the repository I mention just above. You will see the error Error: an error on the server ("") has prevented the request from succeeding (post deployments.apps)
  2. If you execute the command helm upgrade --install movies chart --set tag=latest (be aware of the kubeconfig used), you will see a completely different error

Expected behavior
The error in the Okteto UI should be the same than in the terminal

Desktop (please complete the following information):

Additional context

@ifbyol ifbyol added bug Something isn't working customer reported labels May 9, 2022
@jmacelroy jmacelroy modified the milestones: 2.6, 2.5 May 10, 2022
@jmacelroy jmacelroy removed the triage label May 10, 2022
@AdrianPedriza AdrianPedriza self-assigned this Jun 30, 2022
@AdrianPedriza
Copy link
Contributor

considering that the error if we execute okteto deploy from the terminal no longer offers any panic, but:

Captura de pantalla 2022-06-30 a las 15 27 45

the error returned by the UI is Error: an error on the server ("") has prevented the request from succeeding (post deployments.apps)

and the error returned by execute helm upgrade --install movies chart --set tag=latest is:

Captura de pantalla 2022-06-30 a las 16 01 10

@ifbyol do we want our error message (in UI and CLI) to be directly the one returned by the helm command (last image) or do we want the UI to contain the error message currently displayed by the CLI (Error executing command 'helm upgrade --install movies chart --set tag=${OKTETO_GIT_COMMIT}': exit status 1)?

@ifbyol
Copy link
Member Author

ifbyol commented Jun 30, 2022

considering that the error if we execute okteto deploy from the terminal no longer offers any panic, but:

Captura de pantalla 2022-06-30 a las 15 27 45

The panic is thrown by helm command when executing the upgrade, not the okteto deploy

the error returned by the UI is Error: an error on the server ("") has prevented the request from succeeding (post deployments.apps)

and the error returned by execute helm upgrade --install movies chart --set tag=latest is:

Captura de pantalla 2022-06-30 a las 16 01 10

@ifbyol do we want our error message (in UI and CLI) to be directly the one returned by the helm command (last image) or do we want the UI to contain the error message currently displayed by the CLI (Error executing command 'helm upgrade --install movies chart --set tag=${OKTETO_GIT_COMMIT}': exit status 1)?

We want the helm error as is the one which contain the information about the real error. It is hard to debug the real issue if we only have this Error: an error on the server ("") has prevented the request from succeeding (post deployments.apps)

@AdrianPedriza
Copy link
Contributor

After investigating the causes of this issue we came to the conclusion that we do not get the same output since our proxy previously detects the expected error in this line which means that the request is not forwarded to the cluster (with the added label) and as a consequence we never get the expected error.

If we want to get the same error we should forward the request to the cluster, even if we are not able to add the label. If we act in this way, we must be aware that if there is no error in the cluster and we have not modified the resource in the proxy, we will have resources in the cluster without the expected labels.

@pchico83 @jmacelroy What do you think is the best solution in this case?

@ifbyol
Copy link
Member Author

ifbyol commented Jul 6, 2022

I would go with forwarding the request to the server. We try to convert the body to the spec of the resource. If it fails, it would probably fail in the server. And if it doesn't fail, we don't block the user. If they have to delete some resources manually, I think it is better than preventing them to deploy the application.

We might have issues in the future if some new version with different spec fields is released as we don't check the API version before trying to convert it to a specific spec, but I think that is another issue to be handled in another ticket. I'll create an issue for it

@pchico83
Copy link
Contributor

pchico83 commented Jul 6, 2022

+1 for @ifbyol approach

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working customer reported
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants