diff --git a/README.md b/README.md index 734032c2..b05f9610 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,9 @@ The documentation for this project can be found [here](https://ibm.github.io/Ansible-OpenShift-Provisioning/). ## What's new: +* Added support for LPAR based installation. The process for installing based on LPAR will largely use the same playbooks with some modifications to the configuration which can be found in the updated docs. +* Infra nodes on LPAR is currently unsupported, will be added in a later patch. +* Installation with bastion and bootstrap as KVM nodes and also extra worker or compute nodes as KVM along with lpar nodes is supported. ### Variables renamed: diff --git a/docs/set-variables-group-vars.md b/docs/set-variables-group-vars.md index 0be35045..f214c79e 100644 --- a/docs/set-variables-group-vars.md +++ b/docs/set-variables-group-vars.md @@ -10,6 +10,7 @@ ## 1 - Controller **Variable Name** | **Description** | **Example** :--- | :--- | :--- +**env.installation_type** | Can be of type kvm or lpar. Some packages will be ignored for installation in case of non lpar based installation. | kvm **env.controller.sudo_pass** | The password to the machine running Ansible (localhost).
This will only be used for two things. To ensure you've installed the
pre-requisite packages if you're on Linux, and to add the login URL
to your /etc/hosts file. | Pas$w0rd! ## 2 - LPAR(s) @@ -41,6 +42,7 @@ **env.file_server.user** | Username to connect to the file server. Must have sudo and SSH access. | user1 **env.file_server.pass** | Password to connect to the file server as above user. | user1pa$s! **env.file_server.protocol** | Protocol used to serve the files, either 'ftp' or 'http' | http +**env.file_server.iso_os_variant** | The os variant for the bastion kvm to be created | rhel8.8 **env.file_server.iso_mount_dir** | Directory path relative to the HTTP/FTP accessible directory where RHEL ISO is mounted. For example, if the FTP root is at /home/user1
and the ISO is mounted at /home/user1/RHEL/8.7 then this variable would be
RHEL/8.7 - no slash before or after. | RHEL/8.7 **env.file_server.cfgs_dir** | Directory path relative to to the HTTP/FTP accessible directory where configuration files can be stored. For example, if FTP root is /home/user1
and you would like to store the configs at /home/user1/ocpz-config then this variable would be
ocpz-config. No slash before or after. | ocpz-config diff --git a/docs/set-variables-host-vars.md b/docs/set-variables-host-vars.md index 1c22f434..549e516a 100644 --- a/docs/set-variables-host-vars.md +++ b/docs/set-variables-host-vars.md @@ -15,8 +15,11 @@ :--- | :--- | :--- **networking.hostname** | The hostname of the LPAR with RHEL installed natively (the KVM host). | kvm-host-01 **networking.ip** | The IPv4 address of the LPAR with RHEL installed natively (the KVM host). | 192.168.10.2 +**networking.ipv6** | IPv6 address for the bastion if use_ipv6 variable is 'True'. | fd00::3 **networking.subnetmask** | The subnet that the LPAR resides in within your network. | 255.255.255.0 **networking.gateway** | The IPv4 address of the gateway to the network where the KVM host resides. | 192.168.10.0 +**networking.ipv6_gateway** | IPv6 of he bastion's gateway server. | fd00::1 +**networking.ipv6_prefix** | IPv6 prefix. | 64 **networking.nameserver1** | The IPv4 address from which the KVM host gets its hostname resolved. | 192.168.10.200 **networking.nameserver2** | (Optional) A second IPv4 address from which the KVM host can get its hostname
resolved. Used for high availability. | 192.168.10.201 **networking.device1** | The network interface card from Linux's perspective. Usually enc and then a number that comes
from the dev_num of the network adapter. | enc100 @@ -25,8 +28,11 @@ ## Important Note * You can skip the rest of the variables on this page IF you are using existing LPAR(s) that has RHEL already installed. +* If you are installing an LPAR based cluster then the information below must be provided and are not optional. You must create a host file corresponding to each lpar node. * Since this is how most production deployments on-prem are done on IBM zSystems, these variables have been marked as optional. * With pre-existing LPARs with RHEL installed, you can also skip [1_create_lpar.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/1_create_lpar.yaml) and [2_create_kvm_host.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/2_create_kvm_host.yaml) playbooks. Make sure to still do [0_setup.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/0_setup.yaml) first though, then skip to [3_setup_kvm_host.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/3_setup_kvm_host.yaml) + * In the scenario of lpar based installation you can skip [1_create_lpar.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/1_create_lpar.yaml) and [2_create_kvm_host.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/2_create_kvm_host.yaml). You can also optionally skip [3_setup_kvm_host.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/3_setup_kvm_host.yaml) and [4_create_bastion.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/3_setup_kvm_host.yaml) unless you are planning on having the bastion on the same host. + * In case of lpar based installation one is expected to have a tessia live disk accessible by the lpar nodes for network boot. The details of which are to be filled in section #7 below. The steps to create a tessia livedisk can be found [here](https://gitlab.com/tessia-project/tessia-baselib/-/blob/master/doc/users/live_image.md). ## 2 - (Optional) CPC & HMC **Variable Name** | **Description** | **Example** @@ -81,4 +87,11 @@ **lpar.storage_group_2.type** | (Optional) Storage type. FCP is the only tested type as of now. | fcp **lpar.storage_group_2_.storage_wwpn** | (Optional) World-wide port numbers for storage group. Use provided list formatting. | 500708680235c3f0
500708680235c3f1
500708680235c3f2
500708680235c3f3 **lpar.storage_group_2_.dev_num** | (Optional) The logical device number of the Host Bus Adapter (HBA) for the storage group. | C001 -**lpar.storage_group_2_.lun_name** | (Optional) he Logical Unit Numbers (LUN) that points to a specific virtual disk
behind the WWPN. | 4200569309ahhd240000000000000c001 \ No newline at end of file +**lpar.storage_group_2_.lun_name** | (Optional) he Logical Unit Numbers (LUN) that points to a specific virtual disk
behind the WWPN. | 4200569309ahhd240000000000000c001 + +## 7 - (Optional) Livedisk info +**Variable Name** | **Description** | **Example** +:--- | :--- | :--- +**lpar.livedisk.livedisktype** | (Optional) Storage type. DASD is the only tested type as of now. | dasd +**lpar.livedisk.devicenr** | (Optional) the device no of the DASD live disk | c6h1 +**lpar.livedisk.livedisk_root_pass** | (Optional) root password for the livedisk | p@ssword diff --git a/inventories/default/group_vars/all.yaml.template b/inventories/default/group_vars/all.yaml.template index 6924d309..d1ad6380 100644 --- a/inventories/default/group_vars/all.yaml.template +++ b/inventories/default/group_vars/all.yaml.template @@ -5,6 +5,7 @@ # Section 1 - Ansible Controller env: + installation_type: #X controller: sudo_pass: #X @@ -479,7 +480,7 @@ day2_compute_node: # Section 19 - Agent Based Installer ( Optional ) abi: - flag: True + flag: Flase ansible_workdir: 'ansible_workdir' ocp_installer_version: '4.15.0-rc.8' ocp_installer_url: 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/' diff --git a/inventories/default/host_vars/KVMhostname1-here.yaml.template b/inventories/default/host_vars/KVMhostname1-here.yaml.template index c4aa3d1c..b53a9cc5 100644 --- a/inventories/default/host_vars/KVMhostname1-here.yaml.template +++ b/inventories/default/host_vars/KVMhostname1-here.yaml.template @@ -2,8 +2,11 @@ networking: hostname: #X ip: #X + ipv6: #X subnetmask: #X + ipv6_prefix: #X gateway: #X + ipv6_gateway: #X nameserver1: #X # nameserver2: device1: #X @@ -14,9 +17,11 @@ storage: ############################################################## # Variables below this point only need to be filled out if # -# env.z.lpar1.create is True. Meaning, if the LPARs you will # +# env.z.lpar1.create is True or if you are doing a lpar # +# based installation. Meaning, if the LPARs you will # # be using as KVM host(s) already exist and have RHEL # -# installed, the variables below are not required. # +# installed, the variables below are not required. However # +# for lpar based installation you must provide below vars # ############################################################## # Section 2 - CPC & HMC @@ -81,4 +86,10 @@ lpar: # - # - # dev_num: -# lun_name: \ No newline at end of file +# lun_name: + +# Section 7 -live disk info + livedisk: + livedisktype: #X + devicenr: #X + livedisk_root_pass: #X \ No newline at end of file diff --git a/inventories/default/host_vars/KVMhostname2-here.yaml.template b/inventories/default/host_vars/KVMhostname2-here.yaml.template index 89b801f5..8a5dc9bf 100644 --- a/inventories/default/host_vars/KVMhostname2-here.yaml.template +++ b/inventories/default/host_vars/KVMhostname2-here.yaml.template @@ -2,8 +2,11 @@ networking: hostname: #X ip: #X + ipv6: #X subnetmask: #X + ipv6_prefix: #X gateway: #X + ipv6_gateway: #X nameserver1: #X # nameserver2: device1: #X @@ -14,9 +17,11 @@ storage: ############################################################## # Variables below this point only need to be filled out if # -# env.z.lpar2.create is True. Meaning, if the LPARs you will # +# env.z.lpar2.create is True or if you are doing a lpar # +# based installation. Meaning, if the LPARs you will # # be using as KVM host(s) already exist and have RHEL # -# installed, the variables below are not required. # +# installed, the variables below are not required. However # +# for lpar based installation you must provide below vars # ############################################################## # Section 2 - CPC & HMC @@ -81,4 +86,10 @@ lpar: # - # - # dev_num: -# lun_name: \ No newline at end of file +# lun_name: + +# Section 7 -live disk info + livedisk: + livedisktype: #X + devicenr: #X + livedisk_root_pass: #X \ No newline at end of file diff --git a/inventories/default/host_vars/KVMhostname3-here.yaml.template b/inventories/default/host_vars/KVMhostname3-here.yaml.template index 997b7f61..432df5ac 100644 --- a/inventories/default/host_vars/KVMhostname3-here.yaml.template +++ b/inventories/default/host_vars/KVMhostname3-here.yaml.template @@ -2,8 +2,11 @@ networking: hostname: #X ip: #X + ipv6: #X subnetmask: #X + ipv6_prefix: #X gateway: #X + ipv6_gateway: #X nameserver1: #X # nameserver2: device1: #X @@ -14,9 +17,11 @@ storage: ############################################################## # Variables below this point only need to be filled out if # -# env.z.lpar3.create is True. Meaning, if the LPARs you will # +# env.z.lpar3.create is True or if you are doing a lpar # +# based installation. Meaning, if the LPARs you will # # be using as KVM host(s) already exist and have RHEL # -# installed, the variables below are not required. # +# installed, the variables below are not required. However # +# for lpar based installation you must provide below vars # ############################################################## # Section 2 - CPC & HMC @@ -81,4 +86,10 @@ lpar: # - # - # dev_num: -# lun_name: \ No newline at end of file +# lun_name: + +# Section 7 -live disk info + livedisk: + livedisktype: #X + devicenr: #X + livedisk_root_pass: #X \ No newline at end of file diff --git a/playbooks/1_create_lpar.yaml b/playbooks/1_create_lpar.yaml index ffe8df41..9425879b 100644 --- a/playbooks/1_create_lpar.yaml +++ b/playbooks/1_create_lpar.yaml @@ -52,4 +52,7 @@ - name: Create an LPAR for a third KVM host. import_role: name: create_lpar - when: env.z.lpar3.create == True \ No newline at end of file + when: env.z.lpar3.create == True + + +#TODO: create all lpars defined in the host vars section \ No newline at end of file diff --git a/playbooks/5_setup_bastion.yaml b/playbooks/5_setup_bastion.yaml index 80c7e7b2..e81bb83e 100644 --- a/playbooks/5_setup_bastion.yaml +++ b/playbooks/5_setup_bastion.yaml @@ -81,6 +81,7 @@ - { role: haproxy, when: env.bastion.options.loadbalancer.on_bastion } - { role: sno_haproxy, when: env.bastion.options.loadbalancer.on_bastion and control_node_count | int == 1 } - httpd + - { role: install_tessia_baselib, when: env.installation_type|lower == "lpar" } - hosts: bastion tags: services, section_2, openvpn @@ -89,8 +90,8 @@ openvpn_role: "server" roles: #- { role: robertdebock.bootstrap, tags: openvpn, when: env.z.high_availability == True } - - { role: robertdebock.epel, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True } - - { role: robertdebock.openvpn, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True } + - { role: robertdebock.epel, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" } + - { role: robertdebock.openvpn, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" } - hosts: localhost tags: services, section_2, openvpn @@ -101,7 +102,7 @@ file: state: directory path: tmp - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" - hosts: bastion tags: services, section_2, openvpn @@ -118,7 +119,7 @@ - issued/client.crt - private/client.key - ta.key - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" - name: setup OpenVPN on KVM host(s). hosts: kvm_host @@ -131,7 +132,7 @@ pre_tasks: - name: Gather facts. setup: - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" - name: Create landing directories for certificates and keys on KVM hosts. tags: openvpn @@ -142,7 +143,7 @@ loop: - issued - private - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" - name: Copy certificates and keys from controller to KVM hosts. tags: openvpn @@ -155,10 +156,10 @@ - client.crt - client.key - ta.key - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "zvm" and env.installation_type|lower != "lpar" roles: - - { role: robertdebock.epel, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True } - - { role: robertdebock.openvpn, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True } + - { role: robertdebock.epel, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" } + - { role: robertdebock.openvpn, tags: openvpn, when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" } - hosts: localhost tags: services, section_2, openvpn @@ -169,7 +170,7 @@ file: state: absent path: tmp - when: env.setup_openvpn == True and env.z.high_availability == True + when: env.setup_openvpn == True and env.z.high_availability == True and env.installation_type|lower != "lpar" - hosts: bastion tags: get_ocp, section_3 @@ -178,4 +179,4 @@ - "{{ inventory_dir }}/group_vars/all.yaml" roles: - common - - { role: get_ocp, when: abi.flag is defined and abi.flag != True } + - { role: get_ocp, when: abi.flag is not defined or abi.flag != True } diff --git a/playbooks/6_create_nodes.yaml b/playbooks/6_create_nodes.yaml index 278a7272..cb47a179 100644 --- a/playbooks/6_create_nodes.yaml +++ b/playbooks/6_create_nodes.yaml @@ -1,5 +1,20 @@ --- +- name: 6 create nodes - find lpar host files + hosts: localhost + vars_files: + - "{{ inventory_dir }}/group_vars/all.yaml" + tasks: + - name: Loop over node types and include role + include_role: + name: check_for_lpar_nodes + loop: + - bootstrap + - compute + - control + loop_control: + loop_var: node_type + # Prepare and then create the temporary bootstrap node and the control nodes - name: 6 create nodes - prepare KVM guests hosts: kvm_host @@ -7,9 +22,9 @@ vars_files: - "{{ inventory_dir }}/group_vars/all.yaml" roles: - - prep_kvm_guests + - { role: prep_kvm_guests, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } # Delete control, compute and infra nodes, if exists - - delete_nodes + - { role: delete_nodes, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } - name: 6 create nodes - create bootstrap hosts: kvm_host[0] @@ -17,8 +32,32 @@ vars_files: - "{{ inventory_dir }}/group_vars/all.yaml" roles: - - common - - create_bootstrap + - { role: common, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } + - { role: create_bootstrap, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } + +- name: 6 create nodes - bootstrap and control ( if in lpar ) + hosts: bastion + become: true + tasks: + - name: boot bootstrap + vars: + node_type: "bootstrap" + node_name: "{{ item }}" + ignition: "bootstrap" + include_tasks: + file: ../roles/boot_LPAR/tasks/main.yaml + loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}" + when: item in hosts_with_host_vars + + - name: boot control nodes + vars: + node_type: "control" + node_name: "{{ item }}" + ignition: "master" + include_tasks: + file: ../roles/boot_LPAR/tasks/main.yaml + loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}" + when: item in hosts_with_host_vars - name: 6 create nodes - create control nodes hosts: kvm_host @@ -64,6 +103,7 @@ name: "{{ env.cluster.nodes.bootstrap.vm_name }}" command: destroy ignore_errors: true + when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars - name: Undefine bootstrap. Expect ignored errors if bootstrap is already undefined. tags: create_nodes, teardown_bootstrap @@ -71,7 +111,7 @@ name: "{{ env.cluster.nodes.bootstrap.vm_name }}" command: undefine ignore_errors: true - + - name: 6 create nodes - once bootstrapping is complete, create compute nodes. hosts: kvm_host tags: create_compute_nodes @@ -79,5 +119,19 @@ vars_files: - "{{ inventory_dir }}/group_vars/all.yaml" roles: - - common - - create_compute_nodes + - { role: common, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } + - { role: create_compute_nodes, when: env.cluster.nodes.bootstrap.vm_name not in hosts_with_host_vars } + +- name: 6 create nodes - compute ( if in lpar ) + hosts: bastion + become: true + tasks: + - name: boot compute nodes + vars: + node_type: "compute" + node_name: "{{ item }}" + ignition: "worker" + include_tasks: + file: ../roles/boot_LPAR/tasks/main.yaml + loop: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}" + when: item in hosts_with_host_vars diff --git a/roles/boot_LPAR/tasks/main.yaml b/roles/boot_LPAR/tasks/main.yaml new file mode 100644 index 00000000..b70489fe --- /dev/null +++ b/roles/boot_LPAR/tasks/main.yaml @@ -0,0 +1,74 @@ +--- +- name: Boot LPAR with corresponding ignition + block: + - name: set fcp fact to empty + set_fact: rd_zfcp_string='' ipv6_string='' + + - name: inlude host vars for given lpar + include_vars: + file: "{{ inventory_dir }}/host_vars/{{ item }}.yaml" + name: node + + - name: Getting script for booting + template: + src: ../templates/boot_lpar.py + dest: /root/ansible_workdir/boot_lpar.py + + - name: Build rd.zfcp string + set_fact: + rd_zfcp_string: "{{ rd_zfcp_string | default('') }} rd.zfcp=0.0.{{ node.lpar.storage_group_1.dev_num }},{{ wwpn }},{{ node.lpar.storage_group_1.lun_name }}" + loop: "{{ node.lpar.storage_group_1.storage_wwpn }}" + loop_control: + loop_var: wwpn + + - name: set ipv6 string + set_fact: + ipv6_string: "ip=[{{ node.networking.ipv6 }}]::[{{ node.networking.ipv6_gateway }}]:{{ node.networking.ipv6_prefix }}::{{ env.cluster.networking.interface }}:none" + when: env.use_ipv6 == true + + - name: Debug + debug: + msg: + python3 /root/ansible_workdir/boot_lpar.py \ + --cpcname {{ node.cpc_name }} \ + --lparname {{ node.lpar.name }} \ + --hmchost {{ node.hmc.host }} \ + --hmcuser {{ node.hmc.auth.user }} \ + --hmcpass {{ node.hmc.auth.pass }} \ + --cpu {{ node.lpar.ifl.count }} \ + --memory {{ node.lpar.ifl.initial_memory }} \ + --kernel {{ rhcos_download_url }}{{ rhcos_live_kernel }} \ + --initrd {{ rhcos_download_url }}{{ rhcos_live_initrd }} \ + --livedisktype {{ node.lpar.livedisk.livedisktype }} \ + --devicenr {{ node.lpar.livedisk.devicenr }} \ + --netset_ip {{ node.networking.ip }} \ + --netset_gateway {{ node.networking.gateway }} \ + --netset_network_type osa \ + --netset_network_device "{{ node.lpar.networking.nic.card1.dev_num }}" \ + --netset_password {{ node.lpar.livedisk.livedisk_root_pass }} \ + --netset_dns "{{ node.networking.nameserver1 }},{{ node.networking.nameserver2 }}" \ + --log_level DEBUG \ + --cmdline 'rd.neednet=1 console=ttysclp0 coreos.inst.install_dev=sda coreos.live.rootfs_url=http://{{ env.bastion.networking.ip }}:8080/bin/{{ rhcos_live_rootfs }} coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/{{ ignition }}.ign ip={{ node.networking.ip }}::{{ node.networking.gateway }}:{{ node.networking.subnetmask }}:{{ node.networking.hostname }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}:{{ node.networking.device1 }}:none nameserver={{ env.bastion.networking.ip }} cio_ignore=all,!condev zfcp.allow_lun_scan=0 rd.znet=qeth,{{ node.lpar.networking.nic.card1.dev_num }},layer2=1 {{ ipv6_string }} {{ rd_zfcp_string }}' + + - name: Booting lpar node + shell: | + python3 /root/ansible_workdir/boot_lpar.py \ + --cpcname {{ node.cpc_name }} \ + --lparname {{ node.lpar.name }} \ + --hmchost {{ node.hmc.host }} \ + --hmcuser {{ node.hmc.auth.user }} \ + --hmcpass {{ node.hmc.auth.pass }} \ + --cpu {{ node.lpar.ifl.count }} \ + --memory {{ node.lpar.ifl.initial_memory }} \ + --kernel {{ rhcos_download_url }}{{ rhcos_live_kernel }} \ + --initrd {{ rhcos_download_url }}{{ rhcos_live_initrd }} \ + --livedisktype {{ node.lpar.livedisk.livedisktype }} \ + --devicenr {{ node.lpar.livedisk.devicenr }} \ + --netset_ip {{ node.networking.ip }} \ + --netset_gateway {{ node.networking.gateway }} \ + --netset_network_type osa \ + --netset_network_device "{{ node.lpar.networking.nic.card1.dev_num }}" \ + --netset_password {{ node.lpar.livedisk.livedisk_root_pass }} \ + --netset_dns "{{ node.networking.nameserver1 }},{{ node.networking.nameserver2 }}" \ + --log_level DEBUG \ + --cmdline 'rd.neednet=1 console=ttysclp0 coreos.inst.install_dev=sda coreos.live.rootfs_url=http://{{ env.bastion.networking.ip }}:8080/bin/{{ rhcos_live_rootfs }} coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/{{ ignition }}.ign ip={{ node.networking.ip }}::{{ node.networking.gateway }}:{{ node.networking.subnetmask }}:{{ node.networking.hostname }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}:{{ node.networking.device1 }}:none nameserver={{ env.bastion.networking.ip }} cio_ignore=all,!condev zfcp.allow_lun_scan=0 rd.znet=qeth,{{ node.lpar.networking.nic.card1.dev_num }},layer2=1 {{ ipv6_string }} {{ rd_zfcp_string }}' diff --git a/roles/boot_LPAR/templates/boot_lpar.py b/roles/boot_LPAR/templates/boot_lpar.py new file mode 100644 index 00000000..8550b4e3 --- /dev/null +++ b/roles/boot_LPAR/templates/boot_lpar.py @@ -0,0 +1,83 @@ +from tessia.baselib.hypervisors.hmc import hmc +import logging +import argparse + +def list_of_strings(arg): + return arg.split(',') + +parser = argparse.ArgumentParser(description="Get the environment.") + +parser.add_argument("--cpcname", type=str, help="this is the CEC/CPC hosting the target LPAR" ) +parser.add_argument("--lparname", type=str, help="Hypervisor name", required=True) +parser.add_argument("--hmchost", type=str, help="HMC Hostname or IP, enter the hmc url here", required=True) +parser.add_argument("--hmcuser", type=str, help="HMC user", required=True) +parser.add_argument("--hmcpass", type=str, help="HMC user password", required=True) +parser.add_argument("--cpu", type=int, help="number of Guest CPUs", required=True) +parser.add_argument("--memory", type=int, help="Guest memory in MB", required=True) +parser.add_argument("--kernel", type=str, help="kernel URI", required=True, default='') +parser.add_argument("--cmdline", type=str, help="kernel cmdline", required=True, default='') +parser.add_argument("--initrd", type=str, help="Initrd URI", required=True, default='') + +#live disk info +parser.add_argument("--livedisktype", type=str, help="Can be of type dasd or scsi", required=True, default='') +parser.add_argument("--devicenr", type=str, help="deviceenr for the live disk image") +parser.add_argument("--netset_ip", type=str, help="network setup ip for the live image") +parser.add_argument("--netset_gateway", type=str) +parser.add_argument("--netset_network_type", type=str, help="could be of type osa or pci") +parser.add_argument("--netset_network_device", type=str, help="network device id") +parser.add_argument("--netset_password", type=str, help="live disk password") +parser.add_argument("--netset_dns", type=list_of_strings, help="comma seperated list of dns addresss in order") + +parser.add_argument("--log_level", type=str, help="can be of type INFO or DEBUG") +args = parser.parse_args() + + + +# cpc_name = "a46" # this is the CEC/CPC hosting the target LPAR +# hmc_address = "hmca2.boeblingen.de.ibm.com" # URL where HMC API is running +# hmc_user = "" +# hmc_password = "" +# Currently there are no parameters for instantiating a hmc hypervisor +hypervisor_params = None +hmc = hmc.HypervisorHmc(args.cpcname, args.hmchost, + args.hmcuser, args.hmcpass, hypervisor_params) + +# enable below block for debug output +if args.log_level == "DEBUG": + consoleHandler=logging.StreamHandler() + consoleHandler.setLevel(logging.DEBUG) + formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s') + consoleHandler.setFormatter(formatter) + hmc._logger.addHandler(consoleHandler) + hmc._logger.setLevel(logging.DEBUG) +# We must be logged in before submitting any command. +hmc.login() + +# Here we define the parameters of the guest to be started. +lpar_name = args.lparname +lpar_cpu = args.cpu +lpar_memory = args.memory +lpar_parameters = { + "boot_params": { + "boot_method" : args.livedisktype, + "devicenr": args.devicenr, + 'netsetup': { + "mac": None, + "ip": args.netset_ip, + "mask": 16, + "gateway": args.netset_gateway, + "type": args.netset_network_type, # also accepts 'pci' + "device": args.netset_network_device, # enter the function id when type is 'pci' + "password": args.netset_password, + "dns": args.netset_dns, + }, + 'netboot': { + "cmdline": args.cmdline, + "kernel_url": args.kernel, + "initrd_url": args.initrd + } + } +} + +hmc.start(lpar_name, lpar_cpu, lpar_memory, lpar_parameters) +hmc.logoff() \ No newline at end of file diff --git a/roles/check_for_lpar_nodes/tasks/main.yaml b/roles/check_for_lpar_nodes/tasks/main.yaml new file mode 100644 index 00000000..e3356569 --- /dev/null +++ b/roles/check_for_lpar_nodes/tasks/main.yaml @@ -0,0 +1,25 @@ +--- +- name: Initialize variables + set_fact: + hosts_with_host_vars: "{{ hosts_with_host_vars | default([]) }}" + node_list: "{{ q('list',env.cluster.nodes[node_type].vm_name) | flatten }}" + +- name: Check for host_var files + stat: + path: "{{ inventory_dir }}/host_vars/{{ item }}.yaml" + loop: "{{ node_list }}" + register: stat_result + +- name: Update variables based on file existence + set_fact: + hosts_with_host_vars: "{{ hosts_with_host_vars + [item.item] }}" + when: item.stat.exists and item.item not in hosts_with_host_vars + loop: "{{ stat_result.results }}" + +- name: Update group_vars/all.yaml with host variables + lineinfile: + path: "{{ inventory_dir }}/group_vars/all.yaml" + regexp: '^hosts_with_host_vars:' + line: 'hosts_with_host_vars: {{ hosts_with_host_vars }}' + + diff --git a/roles/create_bootstrap/tasks/main.yaml b/roles/create_bootstrap/tasks/main.yaml index 3f8d9e59..592913fc 100644 --- a/roles/create_bootstrap/tasks/main.yaml +++ b/roles/create_bootstrap/tasks/main.yaml @@ -25,7 +25,7 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.bootstrap.ipv6 + ']::[' + env.cluster.networking.ipv6_gateway + ']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/bootstrap.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ diff --git a/roles/create_compute_nodes/tasks/main.yaml b/roles/create_compute_nodes/tasks/main.yaml index 8592cb92..20da3325 100644 --- a/roles/create_compute_nodes/tasks/main.yaml +++ b/roles/create_compute_nodes/tasks/main.yaml @@ -24,7 +24,7 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.compute.ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/worker.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ @@ -36,7 +36,7 @@ loop_control: extended: yes index_var: i - when: env.cluster.nodes.compute.hostname[0] is defined and env.z.high_availability == False and inventory_hostname == env.z.lpar1.hostname + when: env.cluster.nodes.compute.hostname[0] is defined and env.z.high_availability == False and inventory_hostname == env.z.lpar1.hostname and env.cluster.nodes.compute.vm_name[i] not in hosts_with_host_vars - name: Install CoreOS on infra nodes tags: create_compute_nodes @@ -56,7 +56,7 @@ --extra-args "ip={{ env.cluster.nodes.infra.ip[i] }}::{{ env.cluster.networking.gateway }}:{{ env.cluster.networking.subnetmask }}:{{ env.cluster.nodes.infra.hostname[i] }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}:{{ env.cluster.networking.interface }}:none:1500" \ --extra-args "{{ ('ip=[' + env.cluster.nodes.infra.ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/worker.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ @@ -111,7 +111,7 @@ --extra-args "{{ ('ip=[' + compute_ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/worker.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ @@ -122,7 +122,7 @@ loop_control: extended: yes index_var: i - when: env.z.high_availability == True and compute_hostname[i] is defined + when: env.z.high_availability == True and compute_hostname[i] is defined and compute_hostname[i] not in hosts_with_host_vars - name: Create CoreOS infra nodes on KVM hosts, if cluster is to be highly available. tags: create_compute_nodes @@ -142,7 +142,7 @@ --extra-args "ip={{ infra_ip[i] }}::{{ env.cluster.networking.gateway }}:{{ env.cluster.networking.subnetmask }}:{{ infra_hostname[i] }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}:{{ env.cluster.networking.interface }}:none:1500" \ --extra-args "{{ ('ip=[' + infra_ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/worker.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ diff --git a/roles/create_control_nodes/tasks/main.yaml b/roles/create_control_nodes/tasks/main.yaml index b28dbea8..7cdb73ef 100644 --- a/roles/create_control_nodes/tasks/main.yaml +++ b/roles/create_control_nodes/tasks/main.yaml @@ -22,7 +22,7 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.control.ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/master.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ @@ -35,7 +35,7 @@ loop_control: extended: yes index_var: i - when: env.z.high_availability == False and inventory_hostname == env.z.lpar1.hostname + when: env.z.high_availability == False and inventory_hostname == env.z.lpar1.hostname and env.cluster.nodes.control.vm_name[i] not in hosts_with_host_vars # {{ ('ip=[' + env.cluster.nodes.control.ipv6[i] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }} \ - name: Create the first CoreOS control node on the first KVM host, if cluster is to be highly available. @@ -60,14 +60,14 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.control.ipv6[0] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' + env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/master.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ --graphics none \ --wait=-1 \ --noautoconsole - when: env.z.high_availability == True and inventory_hostname == env.z.lpar1.hostname + when: env.z.high_availability == True and inventory_hostname == env.z.lpar1.hostname and env.cluster.nodes.control.vm_name[0] not in hosts_with_host_vars - name: Create the second CoreOS control node on the second KVM host, if cluster is to be highly available. tags: create_control_nodes @@ -91,14 +91,14 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.control.ipv6[1] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/master.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ --graphics none \ --wait=-1 \ --noautoconsole - when: env.z.high_availability == True and inventory_hostname == env.z.lpar2.hostname + when: env.z.high_availability == True and inventory_hostname == env.z.lpar2.hostname and env.cluster.nodes.control.vm_name[1] not in hosts_with_host_vars - name: Create the third CoreOS control node on the third KVM host, if cluster is to be highly available. tags: create_control_nodes @@ -122,11 +122,11 @@ --extra-args "{{ ('ip=[' + env.cluster.nodes.control.ipv6[2] + ']::[' + env.cluster.networking.ipv6_gateway +']:' + env.cluster.networking.ipv6_prefix | string + '::' env.cluster.networking.interface + ':none' ) if env.use_ipv6 == True else '' }}" \ {% endif %} --extra-args "nameserver={{ env.cluster.networking.nameserver1 }}" \ - --extra-args "{{ ('--nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ + --extra-args "{{ ('nameserver=' + env.cluster.networking.nameserver2) if env.cluster.networking.nameserver2 is defined else '' }}" \ --extra-args "coreos.inst.ignition_url=http://{{ env.bastion.networking.ip }}:8080/ignition/master.ign" \ --extra-args "{{ _vm_console }}" \ --memballoon none \ --graphics none \ --wait=-1 \ --noautoconsole - when: env.z.high_availability == True and inventory_hostname == env.z.lpar3.hostname + when: env.z.high_availability == True and inventory_hostname == env.z.lpar3.hostname and env.cluster.nodes.control.vm_name[2] not in hosts_with_host_vars diff --git a/roles/dns/tasks/main.yaml b/roles/dns/tasks/main.yaml index 4e565e16..1775ceb2 100644 --- a/roles/dns/tasks/main.yaml +++ b/roles/dns/tasks/main.yaml @@ -134,7 +134,8 @@ mode: "644" - name: Restart network to update changes made to /etc/resolv.conf - tags: dns, resolv + tags: resolv ansible.builtin.service: name: NetworkManager state: restarted +