[Digital Ocean] Promote to Beta

[srikiz]

Hi,
I wanted thoughts on promoting Digital Ocean to Beta. Currently, a good amount of KOPS functionality is in place for DO, and I wanted to check with the KOPS maintainers if you are okay with this promotion.

Tested with both single cluster and multi-master HA setup
Tested kubernetes upgrade with kops update feature (ex: from v1.19.3 to v1.19.4)
Tested kops validate command on cluster creation
Tested creating multiple instance groups and managing & updating & deleting it.
Tested using the cluster with a DO load balancer when a new service with type: LoadBalancer is used.
Tested creating new PV with CSI drivers for managing DO volumes.

FYI - @timoreimann

[k8s-ci-robot]

Hi @srikiz. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/check-cla

[olemarkus]

I guess a concern from my side is who will own this going forward. I think at least one person who contribute outside of only this provider should commit to also maintain this. Beta is fairly close to stable in our world.

@srikiz the stuff you have been doing here is pretty solid. I don't know if you are interested in contributing to kops as a whole?

[rifelpet]

I would definitely like to see DO promoted to beta. I think its worth clarifying exactly what the criteria is for beta and stable cloud providers. Perhaps we discuss it at the next office hours?

I think it would be great to have automated e2e tests for DO. I believe cluster-api has conformance tests and presubmits running, so I would think much of the setup work is already taken care of if we can get permission to share the DO account with Cluster API. I'm happy to assist with setting up the prow jobs in test-infra once we're ready. I think just periodic tests (similar to GCP's) would be fine.

[srikiz]

I guess a concern from my side is who will own this going forward. I think at least one person who contribute outside of only this provider should commit to also maintain this. Beta is fairly close to stable in our world.

@srikiz the stuff you have been doing here is pretty solid. I don't know if you are interested in contributing to kops as a whole?

Thanks @olemarkus - I would like to contribute to kops as a whole, but currently can't commit to it. May be we can discuss more during our next KOPS meeting.

[timoreimann]

👋 Timo here from DigitalOcean. I've been supporting @srikiz over the last few months as far as support for DigitalOcean in kops is concerned.

We raised the topic on what it takes for a provider to be promoted to beta in a prior office hours meeting on 13. April 2020. Unfortunately, not too many points on that discussion were apparently noted down in the meeting notes. However, I did take notes myself back then and wrote down the following requirements as discussed during the office hours:

• Cluster does not need to be destroyed anymore for rolling upgrade
• Implement validation for DO executed during kops create / update
• HA support

To my understanding, these TODOs are now completed. (@srikiz, correct me if I'm wrong here.)

I think we explicitly excluded test integration at the time. Not exactly sure about the reasons anymore, perhaps the level of support for / ease of adding tests was different back then?
That said, if test integration is expected today from all supported clouds, I'd be happy to talk about it. Let me know what you think, thanks!

[olemarkus]

From my perspective, testing is super nice, but it is not a hard requirement. Timely fixes, keeping up with the development of kops overall etc is more important.

My fear is that DO goes beta, a lot of people starts using it, and then something breaks. Or that we need to update some deps, this breaks the provider, and there is no one around to fix it.

Now @srikiz almost immediately fixed a case of the latter issue earlier today, so I have full confidence that this is not a problem today. I just, perhaps, need a little bit more confidence that this won't be a problem next year.

If DO softly commits to support kops, even support progress DO to stable, then I will happily support DO going to beta in its current state.

[rifelpet]

I tracked down a recording of the office hours in which we discussed DO being promoted to beta: https://youtu.be/A6DXuR4Nu30?t=2584

Justin's points for reaching beta status:

• API Stability
• Going to GA won't require a cluster to be torn down and recreated

@srikiz mentioned an issue around using an internal load balancer for kubelets to reach the apiserver, possibly the useForInternalApi ClusterSpec field. Has that been addressed?

I agree that e2e testing needn't be a hard requirement for beta. I think if we fix the conflicts here and address any review comments then we can bring it up on Friday's office hours call for any final objections and plan to merge it then.

[srikiz]

@rifelpet - I didn't use the useForInternalAPI behavior. As I understand, currently KOPS injects /etc/hosts file entries of all the master hosts and kubelet uses one of those host entries for communicating to api server. DO uses the same behavior in this case.

[rifelpet]

@srikiz ok that makes sense. Feel free to fix the conflicts here and we can discuss and hopefully merge this on Friday.

[rifelpet]

/ok-to-test

[rifelpet]

i think this change should be reverted in order to make the verify-gomod job pass

[rifelpet]

hm not sure why that job failure is still in the GH comment if the job itself passed...

/test pull-kops-verify-packages

[rifelpet]

Everyone on the call today was happy to see DO get promoted to Beta so I'm going to go ahead and merge this 👍🏻 congrats!

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rifelpet, srikiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rifelpet]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[timoreimann]

@rifelpet thanks a lot for your help on driving this one over the finishing line, really appreciate it. Excited to hopefully see more adoption of kops in DigitalOcean over time now that stability / maturity has improved.

[timoreimann]

I have also filed #10386 to track the work needed to get proper e2e testing in place. Sri and I are happy to move that one forward as well.