Refactor to set pullsecret and clusterID #869

praveenkumar · 2019-12-06T13:45:54Z

This refactor allow us to perform more cluster resource changes like
add proxy, insecure registry etc. at cluster level.

Note: Now we are going to depend on the dns even before approving the csr so if local dns is not working then CRC not even able to start.

cfergeau · 2019-12-06T13:50:40Z

pkg/crc/machine/machine.go

+		rerr = m.ToError()
+	}()
+
+	// Update the user pull secret before kubelet start.


Looks like outdated comment? The kubelet was started just before.

@cfergeau By default the pull secret set to null string in our bundle so logically we are updating the pull secret but since on UX side if we say updating the pull secret then user might think that there is already existing pull secret so we just say Adding user's pull secret :)

My problem with this comment is before kubelet start since the code is:

if _, err := sd.Start("kubelet"); err != nil { return fmt.Errorf("Error starting kubelet: %s", err) } //... // Update the user pull secret before kubelet start.

@cfergeau Ah I got it now, I will remove the before kublet start part from comment.

cfergeau · 2019-12-09T07:40:41Z

One thing I forgot to mention, if we start to rely more on the host->VM DNS being functional, then it's a good opportunity to fix #854 to ensure it's there before we try to use it.

praveenkumar · 2019-12-09T08:12:56Z

One thing I forgot to mention, if we start to rely more on the host->VM DNS being functional, then it's a good opportunity to fix #854 to ensure it's there before we try to use it.

@cfergeau yes that's right.

cfergeau

Since I can't easily comment commit by commit... first commit (dns checks) looks good to me.
Second commit, in addition to the comments I made, I believe the commit log could be extended to explain the change that is being made (instead of running commands on the cluster, we now run oc from the host to talk to the cluster).
This refactor allow us to perform ... -> This refactor will allow us to perform.. as the changes which are mentioned are not done in this commit.

cfergeau · 2019-12-10T10:48:56Z

pkg/crc/network/utils.go

+		return err
+	}
+	logging.Debugf("api.crc.testing resolved to %s", ips)
+	ips, err = net.LookupIP(fmt.Sprintf("foo.%s", crcbundleMetadata.ClusterInfo.AppsDomain))


We could be evil and use a randomly generated string to make sure people don't add foo.crc.testing to their /etc/hosts ;) (kidding)

cfergeau · 2019-12-10T10:49:40Z

pkg/crc/network/utils.go

@@ -80,3 +82,17 @@ func DetermineHostIP(instanceIP string) (string, error) {

 	return "", errors.New("unknown error occurred")
 }
+
+func CheckCRCLocalDNSReachableFromHost(crcbundleMetadata *bundle.CrcBundleInfo) error {
+	ips, err := net.LookupIP(fmt.Sprintf("api.%s.%s", crcbundleMetadata.ClusterInfo.ClusterName, crcbundleMetadata.ClusterInfo.BaseDomain))


Does this need to be retried for a few seconds in case the dnsmasq instance takes a bit of time to come up, or is this all synchronous?

@cfergeau since we first check already on the VM which make sure that dnsmasq is running on the VM so on the host it should be good without retry.

cfergeau · 2019-12-10T10:50:35Z

pkg/crc/cluster/cluster.go

+var (
+	// This command make sure we stop the kubelet and clean up the pods
+	// We also providing a 2 sec sleep so that stop pods get settled and
+	// ready for remove. Without this 2 sec time sometime it happens some of


for removal

cfergeau · 2019-12-10T10:50:44Z

pkg/crc/cluster/cluster.go

+
+var (
+	// This command make sure we stop the kubelet and clean up the pods
+	// We also providing a 2 sec sleep so that stop pods get settled and


'stopped pods' ?

and 2 seconds rather than 2 sec, however I don't see a 2 seconds sleep in these crictl commands?
edit: this is managed in the function making use of this variable. I would make this a local variable to that function, this way it's easier to see what the 2 seconds refer to.

cfergeau · 2019-12-10T11:06:09Z

pkg/crc/cluster/cluster.go

+	cmdArgs := []string{"patch", "secret", "pull-secret", "-p",
+		fmt.Sprintf(`{"data":{".dockerconfigjson":"%s"}}`, base64OfPullSec),
+		"-n", "openshift-config", "--type", "merge"}
+	logging.Debugf("Executing oc %s\n", strings.Join(cmdArgs, " "))


This belongs to oc.RunOcCommand rather than here if you want such a log.
logging.Debugf("Patching cluster config for new pull secret") for a debug log which goes in this method.
This is also leaking the pull secret to debug logs.

cfergeau · 2019-12-10T11:06:58Z

pkg/crc/cluster/cluster.go

+		}
+		return nil
+	}
+	return errors.RetryAfter(80, addPullSecret, time.Second)


I'd prefer if we did not have this somehow arbitrary RetryAfter in these methods. A prerequisite for calling this methods is that the api server is up, if needed we can add a method waiting for the api server to be up, and which would contain such a RetryAfter call.

pkg/crc/cluster/cluster.go

cfergeau · 2019-12-12T07:58:07Z

pkg/crc/network/utils.go

+	if err != nil {
+		return err
+	}
+	logging.Debugf("api.crc.testing resolved to %s", ip)


logging.Debugf("api.%s resolved to %s", domainName, ip)

cfergeau · 2019-12-12T07:58:22Z

pkg/crc/network/utils.go

+	if err != nil {
+		return err
+	}
+	logging.Debugf("foo.apps-crc.testing resolved to %s", ip)


logging.Debugf("foo.%s resolved to %s", appsDomainName, ip)
(or somethnig similar :)

cfergeau · 2019-12-12T07:59:06Z

pkg/crc/network/utils.go

@@ -92,3 +94,17 @@ func UriStringForDisplay(uri string) (string, error) {
 	}
 	return uri, nil
 }
+
+func CheckCRCLocalDNSReachableFromHost(crcbundleMetadata *bundle.CrcBundleInfo) error {


Wondering if we need the dependency on the bundlepackage, or if this should be
func CheckCRCLocalDNSReachableFromHost(domainName, appsDomainName string) error ?

cfergeau

Still a couple of minor comments, but overall looks good to me :)

cfergeau · 2019-12-12T08:16:19Z

pkg/crc/machine/machine.go

@@ -682,3 +678,32 @@ func updateSSHKeyPair(sshRunner *crcssh.SSHRunner) error {

 	return err
 }
+
+func configureCluster(ocConfig oc.OcConfig, sshRunner *crcssh.SSHRunner, pullSecret string) (rerr error) {
+	m := errors.MultiError{}


This can be moved in the defer lambda, as it's only used there.

cfergeau · 2019-12-12T08:17:31Z

pkg/crc/machine/machine.go

+		if err := cluster.StopAndRemovePodsInVM(sshRunner); err != nil {
+			m.Collect(err)
+		}
+		rerr = m.ToError()


wondering if we really need a MultiError here though .If sd.Stop() fails, it's very llikely cluster.StopAndRemovePodsInVM will fail too, so we could just report the sd.Stop() error in that case? And if sd.Stop succeeds, then we only have the StopAndRemovePodsInVM error to potentially report.

pkg/crc/cluster/cluster.go

This check will make sure that host can able to resolve the dns for api and app domain of crc using dnsmasq which is running inside the VM.

This refactor will allow us to perform more cluster resource changes like add proxy, insecure registry etc. at cluster level. Now we are running oc commands directly from the host instead running those from the VM.

cfergeau reviewed Dec 6, 2019

View reviewed changes

cfergeau reviewed Dec 10, 2019

View reviewed changes

cfergeau reviewed Dec 12, 2019

View reviewed changes

praveenkumar added 2 commits December 12, 2019 14:22

Issue #854 Add host->cluster DNS checks

ec41d99

This check will make sure that host can able to resolve the dns for api and app domain of crc using dnsmasq which is running inside the VM.

Refactor to set pullsecret and clusterID

075ceaa

This refactor will allow us to perform more cluster resource changes like add proxy, insecure registry etc. at cluster level. Now we are running oc commands directly from the host instead running those from the VM.

cfergeau approved these changes Dec 12, 2019

View reviewed changes

praveenkumar merged commit 866a335 into crc-org:master Dec 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor to set pullsecret and clusterID #869

Refactor to set pullsecret and clusterID #869

praveenkumar commented Dec 6, 2019

cfergeau Dec 6, 2019

praveenkumar Dec 9, 2019

cfergeau Dec 10, 2019

praveenkumar Dec 11, 2019

cfergeau commented Dec 9, 2019

praveenkumar commented Dec 9, 2019

cfergeau left a comment

cfergeau Dec 10, 2019

cfergeau Dec 10, 2019

praveenkumar Dec 10, 2019

cfergeau Dec 10, 2019

cfergeau Dec 10, 2019

cfergeau Dec 10, 2019

cfergeau Dec 10, 2019

cfergeau Dec 10, 2019

cfergeau Dec 12, 2019 •

edited

cfergeau Dec 12, 2019 •

edited

cfergeau Dec 12, 2019

cfergeau left a comment

cfergeau Dec 12, 2019

cfergeau Dec 12, 2019

Refactor to set pullsecret and clusterID #869

Refactor to set pullsecret and clusterID #869

Conversation

praveenkumar commented Dec 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau commented Dec 9, 2019

praveenkumar commented Dec 9, 2019

cfergeau left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau Dec 12, 2019 • edited

Choose a reason for hiding this comment

cfergeau Dec 12, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau Dec 12, 2019 •

edited

cfergeau Dec 12, 2019 •

edited