Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

previewctl install-context --watch #10898

Merged
merged 1 commit into from
Jun 28, 2022
Merged

previewctl install-context --watch #10898

merged 1 commit into from
Jun 28, 2022

Conversation

vulkoingim
Copy link
Contributor

@vulkoingim vulkoingim commented Jun 24, 2022

Description

Implements the --watch flag, using kubeinformers.

Related Issue(s)

Fixes ##10387

How to test

Checkout a new branch:

git checkout -b test-install-ctx

Start a new job with preview

werft run github -j .werft/build.yaml -a with-preview=true

Run previewctl

cd /workspace/gitpod/dev/preview/previewctl
go run main.go install-context --watch
time="2022-06-24 16:51:02" level=info msg="namespace [preview-aa-prev-ctx3] not found" branch=aa/prev-ctx3
time="2022-06-24 16:51:07" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:12" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:17" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:22" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:27" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:32" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:37" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:42" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:44" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:47" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
^[time="2022-06-24 16:51:51" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:51:52" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:51:57" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:52:02" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:52:02" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:52:07" level=info msg="waiting for vm [aa-prev-ctx3] to get ready" branch=aa/prev-ctx3
time="2022-06-24 16:52:08" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:52:08" level=warning msg="error parsing VM IP Addr: vm interfaces length is 0" branch=aa/prev-ctx3
time="2022-06-24 16:52:11" level=info msg="[aa-prev-ctx3] vm ready" branch=aa/prev-ctx3
time="2022-06-24 16:52:16" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:52:32" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:52:49" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:53:05" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:53:21" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:53:37" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:53:53" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:54:09" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3
time="2022-06-24 16:54:25" level=info msg="waiting for context install to succeed" branch=aa/prev-ctx3

Release Notes

NONE

Documentation

Werft options:

  • /werft with-preview

Copy link
Contributor

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Aleks, really brilliant work here ✨! I wasn't expecting that we would get deeper into Kubernetes go-client intricacies so soon with previewctl, but you found a very good use for it with the watch control-loop 🙂. Not only that but you also introduced kubevirt's go-client as well

I believe that the introduction of KubeVirt's lib and k8s informers is great for us. We'll need them for future projects and we can get used to them while previewctl is still simple 😉 With that said, this PR is quite big and my lack of experience with KubeVirt's lib and informers will block me from giving you a super detailed review 😅. I really want to get better at it tho, future reviews will be better! Maybe @liam-j-bennett's experience with k8s controllers could help us here :)


For now, I focused on understanding the changes made to the Preview struct, why they were needed and made some comments about it.

I understand you had to make changes to some commands to make the kubernetes clients work as you expected, but I think the abstractions look a bit weird 🤔.

Would it make sense to split the Kubernetes clients and Preview into separate objects? Something like:

type Harvester struct {
	kClient kubernetes.Interface
	dynamicClient dynamic.Interface
        previews []Preview

	logger *logrus.Entry
}

type Preview struct {
	branch                 string
	name                   string
	namespace              string
        vm                     kubervirtv1.VirtualMachine // Not sure about this one... Maybe we could reuse their library for SSH access? https://github.com/kubevirt/kubevirt/blob/f60c13703d4bc73fe4ebb3c44c6d8379d6ba0577/pkg/virtctl/ssh/ssh.go
}

I really don't want to sound like a blocker here, just trying to surface some ideas to get the abstraction in a good shape so they can be easily enhanced in future interactions :)

dev/preview/previewctl/pkg/preview/preview.go Outdated Show resolved Hide resolved
dev/preview/previewctl/cmd/get_name.go Show resolved Hide resolved
dev/preview/previewctl/cmd/install_context.go Outdated Show resolved Hide resolved
Comment on lines +23 to +28
p, err := preview.New(branch, logger)
if err != nil {
return err
}

err = p.ListAllPreviews()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks a bit strange that in the command get-name you decided to not create a Preview object, but for list you decided to do so 🤨

For me at least I think get-name is correlated to a single Preview and thus creating an object might make sense. list, on the other hand, is not correlated to a single preview and might make sense to not create an object for this command

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ListPreviews you need to talk to the k8s API, and we initialise the configs in the .New(). See my comment for the current preview env.

Comment on lines -18 to +20
Run: func(cmd *cobra.Command, args []string) {
p := preview.New(branch, logger)
RunE: func(cmd *cobra.Command, args []string) error {

err := p.SSHPreview()
err := preview.SSHPreview(branch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as list

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments above for Get and List :)

Comment on lines 24 to 32
_, err := p.harvesterKubeClient.CoreV1().Namespaces().Get(ctx, p.namespace, metav1.GetOptions{})
if err != nil {
return errors.Wrap(errSvcNotReady, err.Error())
}

svc, err := p.harvesterKubeClient.CoreV1().Services(p.namespace).Get(ctx, proxySvcName, metav1.GetOptions{})
if err != nil {
return errors.Wrap(errSvcNotReady, err.Error())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really really like the use of wrapping errors here! 🤌

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Aleks, really brilliant work here ✨! I wasn't expecting that we would get deeper into Kubernetes go-client intricacies so soon with previewctl, but you found a very good use for it with the watch control-loop 🙂. Not only that but you also introduced kubevirt's go-client as well

I believe that the introduction of KubeVirt's lib and k8s informers is great for us. We'll need them for future projects and we can get used to them while previewctl is still simple 😉 With that said, this PR is quite big and my lack of experience with KubeVirt's lib and informers will block me from giving you a super detailed review 😅. I really want to get better at it tho, future reviews will be better! Maybe @liam-j-bennett's experience with k8s controllers could help us here :)

For now, I focused on understanding the changes made to the Preview struct, why they were needed and made some comments about it.

I understand you had to make changes to some commands to make the kubernetes clients work as you expected, but I think the abstractions look a bit weird 🤔.

Would it make sense to split the Kubernetes clients and Preview into separate objects? Something like:

type Harvester struct {
	kClient kubernetes.Interface
	dynamicClient dynamic.Interface
        previews []Preview

	logger *logrus.Entry
}

type Preview struct {
	branch                 string
	name                   string
	namespace              string
        vm                     kubervirtv1.VirtualMachine // Not sure about this one... Maybe we could reuse their library for SSH access? https://github.com/kubevirt/kubevirt/blob/f60c13703d4bc73fe4ebb3c44c6d8379d6ba0577/pkg/virtctl/ssh/ssh.go
}

I really don't want to sound like a blocker here, just trying to surface some ideas to get the abstraction in a good shape so they can be easily enhanced in future interactions :)

Thanks for the feedback! I agree that it would make more sense to have the k8s components split out - I'll do this in a following PR. I also want to change some things around the informers, as currently it's not strictly correct the way they are being used - but I did it this way as it was quicker and it kind kind of grew out a bit more than I intended. Also didn't see the virtctl/ssh package 👍 I think that will make our lives easier. I'll see how it works and add it with the work of porting the install-context :)

@werft-gitpod-dev-com
Copy link

started the job as gitpod-build-aa-prev-ctx.22 because the annotations in the pull request description changed
(with .werft/ from main)

@vulkoingim vulkoingim requested review from a team June 27, 2022 13:02
@github-actions github-actions bot added team: IDE team: delivery Issue belongs to the self-hosted team team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team labels Jun 27, 2022
@github-actions
Copy link
Contributor

⚠️ Hey reviewer! BE CAREFUL ⚠️
Review the code before opening in your Gitpod. .gitpod.yml was changed and it might be harmful.

@vulkoingim vulkoingim removed request for a team June 27, 2022 13:05
Copy link
Contributor

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done!

Let's get this merged and make the improvements you've mentioned in following interactions

@vulkoingim
Copy link
Contributor Author

/unhold

@roboquat roboquat merged commit eee5f2f into main Jun 28, 2022
@roboquat roboquat deleted the aa/prev-ctx branch June 28, 2022 06:18
@roboquat roboquat added deployed: IDE IDE change is running in production deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production labels Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: IDE IDE change is running in production deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production release-note-none size/XXL team: delivery Issue belongs to the self-hosted team team: devx team: IDE team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants