New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let's configure preemptive node pool to run activiti-cloud-acceptance-scenarios #2276

Closed
igdianov opened this Issue Dec 14, 2018 · 1 comment

Comments

3 participants
@igdianov
Copy link
Member

igdianov commented Dec 14, 2018

so that we have enough resources to deploy preview chart for acceptance scenarios in separate node pool with cheap fat preemptive instances and auto-scaling to run acceptance scenarios pipelines

First, we create new node pool called jenkins-x-builders in GKE cluster with Preemptible nodes and Autoscaling options enabled. The node size should be minimum 8 CPU's or bigger to accommodate heavy workloads. We must also add NO_SCHEDULE:gke-preemtible=true taint when creating the node pool in order to avoid scheduling other regular pods on preemptible nodes.

GKE will automatically add cloud.google.com/gke-preemptible label to each node instance so we can use it in our pipeline's Kubernetes agent's customization yamlFile to specify nodeSelector and taint tolerations:

gke-preemtible.yaml:

spec:
  # The node selector says that pod must be scheduled only on nodes that have specific label and value. 
  nodeSelector:
    cloud.google.com/gke-preemptible: "true"
    
  # It is necessary to add toleration to GKE preemtible pool taint to the pod spec in order to schedule it on preemptible nodes
  tolerations:
  - key: "gke-preemptible"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"        

We also need to change Jenkinsfile pipeline with Kubernetes Plugin configuration to use our gke-preemptible customization:

pipeline {
    agent {
       kubernetes {
              // Change the name of jenkins-maven label to be able to use yaml configuration snippet
              label "maven-gke-preemptible"
              // Inherit from Jx Maven pod template
              inheritFrom "maven"
              // Add pod configuration to Jenkins builder pod template
              yamlFile "gke-preemtible.yaml"
            }
    }

Node labels and tolerations for pod assignment reference:

https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/

@salaboy salaboy added this to Open in Activiti 7.x via automation Dec 14, 2018

@salaboy salaboy added this to the RC1 milestone Dec 14, 2018

@almerico

This comment has been minimized.

Copy link
Member

almerico commented Dec 18, 2018

screen shot 2018-12-18 at 9 58 21 am
During configuration stumble to strange limit

@almerico almerico added the blocked label Dec 18, 2018

@almerico almerico closed this Jan 4, 2019

Activiti 7.x automation moved this from Open to Closed Jan 4, 2019

@almerico almerico added the wontfix label Jan 4, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment