Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the correct kubeconfig when running in-cluster #437

Merged
merged 6 commits into from
Dec 9, 2022

Conversation

porridge
Copy link
Member

@porridge porridge commented Dec 2, 2022

What this PR does / why we need it:

  • See the issue for an in-depth description of the problem.
  • This PR is best viewed by looking at individual commits.
  • The first three commits are cleanups and refactorings that should not change the behaviour.
  • The last commit changes the code to no longer short-circuit in the in-cluster case, which fixes the aforementioned issue.

Fixes #393

@porridge
Copy link
Member Author

porridge commented Dec 2, 2022

If someone wants to try this out, see https://github.com/porridge/kuttl/releases/tag/v0.14.0-kubeconfig

@kensipe
Copy link
Member

kensipe commented Dec 7, 2022

thanks @porridge !

Signed-off-by: Marcin Owsiany <porridge@redhat.com>
This does not change kuttl behaviour, since all callers of Harness.Config() do
not look at the config if a non-nil error is returned.

The benefit is less cognitive effort when reading this function.

Signed-off-by: Marcin Owsiany <porridge@redhat.com>
Just to keep Config() more focused and not worry about details such as trying
another namespace.

Signed-off-by: Marcin Owsiany <porridge@redhat.com>
Signed-off-by: Marcin Owsiany <porridge@redhat.com>
@porridge
Copy link
Member Author

porridge commented Dec 8, 2022

(rebased)

@kensipe
Copy link
Member

kensipe commented Dec 8, 2022

ok... this is next up for review. expecting is to complete (and likely merge tomorrow)

@kensipe
Copy link
Member

kensipe commented Dec 8, 2022

@porridge just checking... does this work in cluster? a quick review shows testutils.InClusterConfig() removed. I believe that is need for operator sdk scorecard. We unfortunately don't have tests to confirm that is still working. I will need to dig in but the title of the PR indicates that expectation.

the call could be deeper than what is shown in the PR and I will discover tomorrow

@kensipe
Copy link
Member

kensipe commented Dec 8, 2022

disregard... reviewing the context and issues makes things clear... Dziękuję!

@porridge
Copy link
Member Author

porridge commented Dec 9, 2022

Yes, sorry forgot to mention: I did check that log collection works now, with kuttl running both in a pod and on my workstation.

Copy link
Member

@kensipe kensipe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

left some comments... mostly celebrations to godocs and obvious fixes on err conditions. As mentioned, This solves an issue in one use case and I have concerns regarding the removal of inclusterconfig removal. Additionally the change means kubernetes.go::InClusterConfig is no longer used and can be refactored out. I'll grab that as a cleanup on another PR.

Thanks @porridge ! Ready to merge.

@@ -227,6 +227,8 @@ func (h *Harness) RunTestEnv() (*rest.Config, error) {

// Config returns the current Kubernetes configuration - either from the environment
// or from the created temporary control plane.
// As a side effect, on first successful call this method also writes a kubernetes client config file in YAML format
// to a file called "kubeconfig" in the current directory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

}
if err != nil {
return h.config, err
return nil, err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

}
h.T.Logf("Successful connection to cluster at: %s", h.config.Host)
}

// The creation of the "kubeconfig" is necessary for out of cluster execution of kubectl
// The creation of the "kubeconfig" is necessary for out of cluster execution of kubectl,
// as well as in-cluster when the supplied KUBECONFIG is some *other* cluster.
f, err := os.Create("kubeconfig")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should make this a test temp file / folder... however this is consistent with existing behavior deserves it's own PR if we want it. Thanks for the Godocs! really helpful!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this should be in a temporary file (and created with an appropriate umask BTW), see also this related issue.

}

defer f.Close()

return h.config, testutils.Kubeconfig(h.config, f)
}

func (h *Harness) waitForFunctionalCluster() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this should be in kubernetes.go but it's just a refactor of code in the existing file... looks good!

@@ -253,51 +255,54 @@ func (h *Harness) Config() (*rest.Config, error) {
return nil, err
}
h.config.WarningHandler = rest.NewWarningWriter(os.Stderr, rest.WarningWriterOptions{Deduplicate: true})
inCluster, err := testutils.InClusterConfig()
Copy link
Member

@kensipe kensipe Dec 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm most concern with this removal... but it is likely, based on code review, the issue you are looking to solve. I vaguely remember needing this to satisfy the way operator SDK scorecard uses kuttl.

Signed-off-by: Ken Sipe <kensipe@gmail.com>
@kensipe
Copy link
Member

kensipe commented Dec 9, 2022

changed my mind :) Removing the dead code for inclusterconfig makes sense to include on this PR. merging upon CI build success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

When kuttl runs in a pod, kubectl run by it sees incorrect $KUBECONFIG
2 participants