Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[INFRA-3102] Add a Digital Ocean Kubernetes cluster to ci.jenkins.io #2651

Closed
6 tasks done
Tracked by #2680
jenkins-infra-bot opened this issue Oct 15, 2021 · 2 comments
Closed
6 tasks done
Tracked by #2680

[INFRA-3102] Add a Digital Ocean Kubernetes cluster to ci.jenkins.io #2651

jenkins-infra-bot opened this issue Oct 15, 2021 · 2 comments
Assignees
@lemeurherve
Labels
ci.jenkins.io digitalocean imported-jira-issue Issues imported from the Jira INFRA project, see #9 jira-component:ci.jenkins.io jira-component:digitalocean jira-type:digitalocean terraform
Milestone
Terraform 2021/20...

Comments

@jenkins-infra-bot
Copy link

jenkins-infra-bot commented Oct 15, 2021 •
edited by lemeurherve

Why

  • Ref read the EPIC (aws decrease costs)
  • Nurture our relationship with sponsors who are nice to give us free resources
  • Improve resiliency of our services by using multiple cloud providers

What

Add a secondary Kubernetes cluster hosted in Digital Ocean to provide more compute resource for the "container agent" workload in ci.jenkins.io

How

  • Create a new Terraform project "jenkins-infra/digitalocean" like we did with jenkins-infra/aws
  • Cloud credentials in charts-secrets
  • Config cloud credentials for infra.ci in kubernetes-management
  • Job on infra.ci
  • State
  • Add this cluster to ci.jenkins.io

Notes:

Auto-update of DO kubernetes path version will be activated, see https://registry.terraform.io/providers/digitalocean/digitalocean/latest/docs/resources/kubernetes_cluster#auto-upgrade-example

DOKS notable limits

  • The control plane is not highly available and may be temporarily unavailable during upgrades or maintenance. This does not affect running clusters and does not make the cluster workers or workloads unavailable. To get reduced downtime, you can optionally enable the high-availability control plane setting when creating clusters on DOKS versions beginning with 1.21.3-do.0.
  • A cluster must have at least one worker node and cannot be scaled down to zero worker nodes.

Resource limits

  • Clusters can have up to 512 nodes.
  • A single worker node can have up to 110 pods.
  • All worker nodes for a cluster are provisioned in the same datacenter region.
  • Network throughput is capped at 2 Gbps per worker node.

Known (DOKS) issues

  • The certificate authority, client certificate, and client key data in the kubeconfig.yaml file displayed in the control panel expire every seven days after download. If you use this file, you will need to download a new certificate every week. To avoid this, we strongly recommend using doctl.
  • When you renew a Let’s Encrypt certificate, DOKS gives it a new UUID and automatically updates all annotations in the certificate’s cluster to use the new UUID. However, you must manually update any external configuration files and tools that reference the UUID.

References

Part of #2680

Originally reported by dduportal, imported from: Add a Digital Ocean Kubernetes cluster to ci.jenkins.io
  • status: Open
  • priority: Major
  • resolution: Unresolved
  • imported: 2022/01/10
@jenkins-infra-bot
Copy link
Author

dduportal:

Blocked by INFRA-3132

@lemeurherve lemeurherve self-assigned this Jan 17, 2022
@lemeurherve lemeurherve added in-progress Started progress on the issue terraform labels Jan 17, 2022
@lemeurherve lemeurherve mentioned this issue Jan 27, 2022
Closed
7 tasks
@dduportal dduportal added this to the Terraform 2021/2022 Winter milestone Jan 28, 2022
lemeurherve added a commit to jenkins-infra/kubernetes-management that referenced this issue Jan 28, 2022
@lemeurherve
feat: add Digital Ocean secrets
e54a568 
Followup of jenkins-infra/charts-secrets#23

Part of jenkins-infra/helpdesk#2651
This was referenced Jan 28, 2022
Merged
Merged
lemeurherve added a commit to jenkins-infra/kubernetes-management that referenced this issue Jan 28, 2022
@lemeurherve
feat: add Digital Ocean
07cfcc8 
Part of jenkins-infra/helpdesk#2651

Related to jenkins-infra/digitalocean#1
@lemeurherve lemeurherve mentioned this issue Jan 28, 2022
Merged
lemeurherve added a commit to jenkins-infra/kubernetes-management that referenced this issue Jan 29, 2022
@lemeurherve
feat: add Digital Ocean job to infra.ci (#1965)
1d27cb9 
Part of jenkins-infra/helpdesk#2651
smerle33 pushed a commit to smerle33/kubernetes-management that referenced this issue Feb 10, 2022
@lemeurherve @smerleCB
feat: add Digital Ocean job to infra.ci (jenkins-infra#1965)
1cc845d 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Feb 16, 2022
Merged
lemeurherve pushed a commit to lemeurherve/kubernetes-management that referenced this issue Feb 16, 2022
@lemeurherveCB
feat: add helmfile for Digital Ocean cluster
1526803 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Feb 16, 2022
Merged
lemeurherve pushed a commit to lemeurherve/docker-helmfile that referenced this issue Feb 16, 2022
@lemeurherveCB
feat: add doctl for managing Digital Ocean cluster
5632df3 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Feb 16, 2022
Merged
lemeurherve pushed a commit to lemeurherve/docker-helmfile that referenced this issue Feb 16, 2022
@lemeurherveCB
feat: add doctl for managing Digital Ocean cluster
5a29cf5 
Part of jenkins-infra/helpdesk#2651
dduportal added a commit to jenkins-infra/docker-helmfile that referenced this issue Feb 17, 2022
@lemeurherve @dduportal
feat: add doctl for managing Digital Ocean cluster (#63)
58b72a5 
* feat: add doctl for managing Digital Ocean cluster

Part of jenkins-infra/helpdesk#2651

* fix: typo in doctl archive extension

* Update cst.yml

Co-authored-by: Damien Duportal <damien.duportal@gmail.com>

Co-authored-by: Damien Duportal <damien.duportal@gmail.com>
lemeurherve added a commit to jenkins-infra/kubernetes-management that referenced this issue Feb 17, 2022
@lemeurherve
chore: add KUBECONFIG_DOKS credential
7758f8b 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Feb 17, 2022
Merged
dduportal pushed a commit to jenkins-infra/kubernetes-management that referenced this issue Feb 17, 2022
@lemeurherve
chore: add KUBECONFIG_DOKS credential (#2029)
3f69899 
Part of jenkins-infra/helpdesk#2651
lemeurherve pushed a commit to lemeurherve/kubernetes-management that referenced this issue Feb 17, 2022
@lemeurherveCB
feat: add helmfile for Digital Ocean cluster
a2f08af 
Part of jenkins-infra/helpdesk#2651
lemeurherve pushed a commit to lemeurherve/kubernetes-management that referenced this issue Feb 23, 2022
@lemeurherveCB
chore: add datadog to Digital Ocean cluster
bcbd6eb 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Feb 23, 2022
Merged
lemeurherve added a commit to jenkins-infra/kubernetes-management that referenced this issue Feb 23, 2022
@lemeurherve
chore: add datadog to Digital Ocean cluster (#2048)
3bffb7a 
Part of jenkins-infra/helpdesk#2651
@dduportal
Copy link
Contributor

Closing: it works very well, good job!

lemeurherve added a commit to jenkins-infra/jenkins.io that referenced this issue Mar 2, 2022
@lemeurherve
Add Digital Ocean to the sponsors list
0a7a508 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve mentioned this issue Mar 2, 2022
Merged
MarkEWaite pushed a commit to jenkins-infra/jenkins.io that referenced this issue Mar 2, 2022
@lemeurherve
Add Digital Ocean to the sponsors list (#4956)
0f23f9c 
Part of jenkins-infra/helpdesk#2651
@lemeurherve lemeurherve removed the in-progress Started progress on the issue label Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lemeurherve lemeurherve

Labels
ci.jenkins.io digitalocean imported-jira-issue Issues imported from the Jira INFRA project, see #9 jira-component:ci.jenkins.io jira-component:digitalocean jira-type:digitalocean terraform
Projects
None yet
Milestone
Terraform 2021/2022 Winter
Development

No branches or pull requests
3 participants
@dduportal @jenkins-infra-bot @lemeurherve