-
Notifications
You must be signed in to change notification settings - Fork 41
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Changes to handle creation of AgentServiceConfig, Hosted Control Plan…
…e and InfraEnv for agent based hypershift installation (#119) Installs prerequisites on kvm host and will Create AgentServiceConfig, Hosted Control Plane and InfraEnv for the agent based hypershift installation --------- Signed-off-by: DAMISETTI-VEERABHADRARAO <damisetti.veerabhadrarao@ibm.com> Co-authored-by: Jacob Emery <jacob.emery@ibm.com>
- Loading branch information
1 parent
ad88e05
commit 3484455
Showing
12 changed files
with
547 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| # Run the Playbooks | ||
| ## Prerequisites | ||
| * Running OCP Cluster ( Management Cluster ) | ||
| * Multi Cluster Engine (MCE) Operator installed on Management Cluster. | ||
| * MCE instance created and hypershift-preview component enabled. | ||
| * KVM host with root user access | ||
|
|
||
| ## Initial Setup for Hypershift | ||
| * Navigate to the [root folder of the cloned Git repository](https://github.com/IBM/Ansible-OpenShift-Provisioning) in your terminal (`ls` should show [ansible.cfg](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/ansible.cfg)). | ||
|
|
||
| * First playbook to be run is setup_for_hypershift.yaml which will create inventory file for hypershift and will add ssh key to the kvm host. | ||
| ###### Note: | ||
| * If you are running this first time, it will prompt for the password for kvm host for the selected user. | ||
| * Enter password of kvm host to establish SSH key-based authentication. | ||
| * Run this shell command: | ||
| ``` | ||
| ansible-playbook playbooks/setup_for_hypershift.yaml | ||
| ``` | ||
|
|
||
| ## Setup Ansible Vault for Management Cluster Credentials | ||
| ### Overview | ||
| * Creating an encrypted file for storing Management Cluster Credentials. | ||
| ### Steps: | ||
| * The ansible-vault create command is used to create the encrypted file. | ||
| * Create an encrypted file in playbooks directory and set the Vault password ( Below command will prompt for setting Vault password). | ||
| ``` | ||
| ansible-vault create playbooks/secrets.yaml | ||
| ``` | ||
|
|
||
| * Give the credentials of Management Cluster in the encrypted file (created above) in following format. | ||
| ``` | ||
| api_server: '<api-server-url>:<port>' | ||
| user_name: '<username>' | ||
| password: '<password>' | ||
| ``` | ||
|
|
||
| * You can edit the encrypted file using below command | ||
| ``` | ||
| ansible-vault edit playbooks/secrets.yaml | ||
| ``` | ||
| * Make sure you entered Manamegement cluster credenitails properly ,incorrect credentails will cause problem while logging in to the cluster in further steps. | ||
|
|
||
| ## Create Hosted Cluster | ||
| * Here is the playbook which handle the creation of Hosted Cluster using Hypershift , full descriptions of each can be found further down the page. | ||
| * create_hosted_cluster.yaml ([code](https://github.com/veera-damisetti/Ansible-OpenShift-Provisioning/blob/main/playbooks/create_hosted_cluster.yaml)) | ||
| * Run this shell command to run the create_hosted_cluster.yaml playbook: | ||
| ``` | ||
| ansible-playbook playbooks/create_hosted_cluster.yaml | ||
| ``` | ||
|
|
||
|
|
||
| * Watch Ansible as it completes the installation, correcting errors if they arise. | ||
| * To look at what tasks are running in detail, open the playbook or roles/role-name/tasks/main.yaml | ||
|
|
||
| # Description for Playbooks | ||
|
|
||
| ## setup_for_hypershift Playbook | ||
| ### Overview | ||
| * First-time setup of the Ansible Controller,the machine running Ansible. | ||
| ### Outcomes | ||
| * Inventory file for hypershift to be created. | ||
| * SSH key generated for Ansible passwordless authentication. | ||
| * Ansible SSH key is copied to kvm host. | ||
| ### Notes | ||
| * You can use an existing SSH key as your Ansible key, or have Ansible create one for you. | ||
|
|
||
| ## create_hosted_cluster Playbook | ||
| ### Overview | ||
| * Creating AgentServiceConfig, HostedControlPlane , InfraEnv Resources | ||
| ### Outcomes | ||
| * Log in to Management Cluster | ||
| * Creates AgentServiceConfig resource and required configmaps. | ||
| * Deploys HostedControlPlane and . | ||
| * Creates InfraEnv resource and wait till ISO generation. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| --- | ||
|
|
||
| - name: Install Prerequisites on kvm_host | ||
| hosts: kvm_host_hypershift | ||
| vars_files: | ||
| - "{{playbook_dir}}/secrets.yaml" | ||
|
|
||
| roles: | ||
| - install_prerequisites_host_hypershift | ||
|
|
||
|
|
||
|
|
||
| - name: Create AgentServiceConfig Hosted Control Plane and InfraEnv | ||
| hosts: kvm_host_hypershift | ||
| vars_files: | ||
| - "{{playbook_dir}}/secrets.yaml" | ||
| roles: | ||
| - create_agentserviceconfig_hypershift | ||
| - create_hcp_InfraEnv_hypershift | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
|
|
||
| #---------------------------------------------------------- | ||
|
|
||
| - name: Setup on localhost | ||
| hosts: localhost | ||
| roles: | ||
| - create_inventory_setup_hypershift |
49 changes: 49 additions & 0 deletions
49
roles/create_agentserviceconfig_hypershift/tasks/main.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| --- | ||
|
|
||
| - name: Get OCP Release Version | ||
| shell: curl -s {{ hypershift.asc.url_for_ocp_release_file }} | awk '/machine-os / { print $2 }' | ||
| register: ocp_release_version | ||
|
|
||
|
|
||
| - name: Create Config map mirror-config ( For updating AgentServiceConfig with the brew mirror information ) | ||
| template: | ||
| src: mirror-config.yml.j2 | ||
| dest: /root/ansible_workdir/mirror-config.yaml | ||
|
|
||
| - name: Deploy Config map - mirror config | ||
| shell: oc apply -f /root/ansible_workdir/mirror-config.yaml | ||
|
|
||
| - name: Create agenterviceconfig.yaml | ||
| template: | ||
| src: agent_service_config.yaml.j2 | ||
| dest: /root/ansible_workdir/agentserviceconfig.yaml | ||
|
|
||
| - name: Deploy AgentServiceConfig | ||
| command: oc apply -f /root/ansible_workdir/agentserviceconfig.yaml | ||
|
|
||
|
|
||
| - name: Create Config map for ISO_IMAGE_TYPE | ||
| k8s: | ||
| definition: | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: my-assisted-service-config | ||
| namespace: "{{ hypershift.asc.mce_namespace }}" | ||
| data: | ||
| ISO_IMAGE_TYPE: 'full' | ||
|
|
||
| - name: Add the annotation to the AgentServiceConfig | ||
| shell: oc annotate --overwrite AgentServiceConfig agent unsupported.agent-install.openshift.io/assisted-service-configmap=my-assisted-service-config | ||
|
|
||
| - name: Rollout new deployment for assisted-service | ||
| command: oc rollout restart deployment/assisted-service -n {{ hypershift.asc.mce_namespace }} | ||
|
|
||
|
|
||
|
|
||
| - name: Wait for Agent Service Deployment to be Succeeded | ||
| shell: oc get AgentServiceConfig agent -o json | jq -r '.status|.conditions[]|.status' | grep False | wc -l | ||
| register: asc | ||
| until: asc.stdout == '0' | ||
| retries: 60 | ||
| delay: 20 |
25 changes: 25 additions & 0 deletions
25
roles/create_agentserviceconfig_hypershift/templates/agent_service_config.yaml.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| apiVersion: agent-install.openshift.io/v1beta1 | ||
| kind: AgentServiceConfig | ||
| metadata: | ||
| name: agent | ||
| spec: | ||
| mirrorRegistryRef: | ||
| name: mirror-config | ||
| databaseStorage: | ||
| accessModes: | ||
| - ReadWriteOnce | ||
| resources: | ||
| requests: | ||
| storage: "{{ hypershift.asc.db_volume_size}}" | ||
| filesystemStorage: | ||
| accessModes: | ||
| - ReadWriteOnce | ||
| resources: | ||
| requests: | ||
| storage: "{{ hypershift.asc.fs_volume_size }}" | ||
| osImages: | ||
| - openshiftVersion: "{{ hypershift.asc.ocp_version }}" | ||
| version: "{{ ocp_release_version.stdout_lines[0] }}" | ||
| url: "{{ hypershift.asc.iso_url }}" | ||
| rootFSUrl: "{{ hypershift.asc.root_fs_url }}" | ||
| cpuArchitecture: "{{ hypershift.hcp.arch }}" |
32 changes: 32 additions & 0 deletions
32
roles/create_agentserviceconfig_hypershift/templates/mirror-config.yml.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: mirror-config | ||
| namespace: "{{ hypershift.asc.mce_namespace }}" # please verify that this namespace is where MCE is installed. | ||
| labels: | ||
| app: assisted-service | ||
| data: | ||
| registries.conf: | | ||
| unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] | ||
|
|
||
| [[registry]] | ||
| location = "registry.stage.redhat.io" | ||
| insecure = false | ||
| blocked = false | ||
| mirror-by-digest-only = true | ||
| prefix = "" | ||
|
|
||
| [[registry.mirror]] | ||
| location = "brew.registry.redhat.io" | ||
| insecure = false | ||
|
|
||
| [[registry]] | ||
| location = "registry.redhat.io/multicluster-engine" | ||
| insecure = false | ||
| blocked = false | ||
| mirror-by-digest-only = true | ||
| prefix = "" | ||
|
|
||
| [[registry.mirror]] | ||
| location = "brew.registry.redhat.io/multicluster-engine" | ||
| insecure = false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| --- | ||
|
|
||
| - name: Getting Hosted Control Plane Namespace | ||
| set_fact: | ||
| hosted_control_plane_namespace: "{{ hypershift.hcp.clusters_namespace }}-{{ hypershift.hcp.hosted_cluster_name }}" | ||
|
|
||
| - name: Check if Hosted Control Plane Namespace exists | ||
| k8s_info: | ||
| api_version: v1 | ||
| kind: Namespace | ||
| name: "{{ hosted_control_plane_namespace }}" | ||
| register: namespace_check | ||
| ignore_errors: yes | ||
|
|
||
| - name: Create Hosted Control Plane Namespace | ||
| k8s: | ||
| api_version: v1 | ||
| kind: Namespace | ||
| name: "{{ hosted_control_plane_namespace }}" | ||
| state: present | ||
| when: namespace_check.resources | length == 0 | ||
|
|
||
| - name: Get ssh key for host | ||
| slurp: | ||
| src: "~/.ssh/{{ env.ansible_key_name }}.pub" | ||
| register: remote_content_encoded | ||
|
|
||
|
|
||
| - name: Decode the ssh key | ||
| set_fact: | ||
| ssh_key: "{{remote_content_encoded.content | b64decode |replace('\n','')}}" | ||
|
|
||
| - name: Create a Hosted Cluster | ||
| command: > | ||
| hypershift create cluster agent | ||
| --name={{ hypershift.hcp.hosted_cluster_name }} | ||
| --pull-secret={{ hypershift.hcp.pull_secret_file }} | ||
| --agent-namespace={{ hosted_control_plane_namespace }} | ||
| --namespace={{ hypershift.hcp.clusters_namespace }} | ||
| --base-domain= {{ hypershift.hcp.basedomain }} | ||
| --api-server-address=api.{{ hypershift.hcp.hosted_cluster_name }}.{{ hypershift.hcp.basedomain }} | ||
| --ssh-key ~/.ssh/{{ env.ansible_key_name }}.pub | ||
| --release-image=quay.io/openshift-release-dev/ocp-release:{{ hypershift.hcp.ocp_release }} | ||
| - name: Waiting for Hosted Control Plane to be available | ||
| command: oc wait --timeout=30m --for=condition=Available --namespace={{ hypershift.hcp.clusters_namespace }} hostedcluster/{{ hypershift.hcp.hosted_cluster_name }} | ||
|
|
||
| - name: Wait for pods to come up in Hosted Cluster Namespace | ||
| shell: oc get pods -n {{ hosted_control_plane_namespace }} | wc -l | ||
| register: pod_count | ||
| until: pod_count.stdout > '25' | ||
| retries: 40 | ||
| delay: 10 | ||
|
|
||
|
|
||
| - name: Wait for all pods to be in Running State in Hosted Cluster Namespace | ||
| shell: oc get pods -n {{ hosted_control_plane_namespace }} --no-headers | grep -v 'Running\|Completed\|Terminating' | wc -l | ||
| register: pod_status | ||
| until: pod_status.stdout == '0' | ||
| retries: 30 | ||
| delay: 10 | ||
|
|
||
| - name: Create InfraEnv Resource | ||
| k8s: | ||
| namespace: "{{ hosted_control_plane_namespace }}" | ||
| definition: | ||
| apiVersion: agent-install.openshift.io/v1beta1 | ||
| kind: InfraEnv | ||
| metadata: | ||
| name: "{{ hypershift.hcp.hosted_cluster_name }}" | ||
| spec: | ||
| cpuArchitecture: "{{ hypershift.hcp.arch }}" | ||
| pullSecretRef: | ||
| name: pull-secret | ||
| sshAuthorizedKey: "{{ssh_key}}" | ||
|
|
||
|
|
||
|
|
||
| - name: Wait for ISO to generate in InfraEnv | ||
| shell: oc get InfraEnv -n {{ hosted_control_plane_namespace }} --no-headers | ||
| register: infra | ||
| until: infra.stdout.split(' ')[-1] != '' | ||
| retries: 60 | ||
| delay: 20 |
Oops, something went wrong.