fix: Change ftp server to file server (#102)

Related to Issue #96 

The name 'FTP server' is confusing since the KVM host is now capable of
hosting and serving necessary files via http only. Update the docs and
variables to represent this change.

Also, enable using either ftp or http to serve config files for the
bastion. FTP is still the only option for booting the KVM hosts, as that
is the only option for the HMC to network boot from. And HTTP will still
be installed and used on the bastion as it is required. But with these
changes, someone using a pre-existing KVM host and NAT-based networking
can use only HTTP if they'd like. And someone using an FTP server for
booting of the KVM host(s) can use that same FTP server to boot the
bastion.

---------

Signed-off-by: Jacob Emery <jacob.emery@ibm.com>

docs/prerequisites.md

@@ -5,54 +5,81 @@
 * [License](https://access.redhat.com/products/red-hat-openshift-container-platform/) or [free trial](https://www.redhat.com/en/technologies/cloud-computing/openshift/try-it) of Red Hat OpenShift Container Platform for IBM Z systems - s390x architecture (comes with the required licenses for Red Hat Enterprise Linux (RHEL) and CoreOS)
 ## IBM zSystems
 * Hardware Management Console (HMC) access on IBM zSystems or LinuxONE
-* Must be in Dynamic Partition Manager (DPM) mode in order to use the [playbook](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/create_kvm_host.yaml) that automates the creation of the KVM host. 
-    * If DPM mode is not an option for your environment, that playbook can be skipped, but a bare-metal RHEL server must be set-up on an LPAR manually (Filipe Miranda's [how-to article](https://www.linkedin.com/pulse/demystifying-install-process-red-hat-enterprise-linux-filipe-miranda/)) before moving on. Once that is done, continue with the [playbook](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/setup_kvm_host.yaml) that sets up the KVM host.
+* In order to use the [playbook](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/2_create_kvm_host.yaml) that automates the creation of the KVM host Dynamic Partition Manager (DPM) mode is required.
+    * If DPM mode is not an option for your environment, that playbook can be skipped, but a bare-metal RHEL server must be set-up on an LPAR manually (Filipe Miranda's [how-to article](https://www.linkedin.com/pulse/demystifying-install-process-red-hat-enterprise-linux-filipe-miranda/)) before moving on. Once that is done, continue with the [playbook 3](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/3_setup_kvm_host.yaml) that sets up the KVM host.
 * For a minimum installation, at least:
     * 6 Integrated Facilities for Linux (IFLs) with SMT2 enabled
     * 85 GB of RAM
     * An FCP storage group created with 1 TB of disk space
     * 8 IPv4 addresses
-## FTP Server
-* On the same network as your IBM zSystems / LinuxONE hardware.
-* Red Hat Enterprise Linux (RHEL) 8 for s390x architecture mounted in an accessible folder (e.g. /home/ftpuser/rhel/)
-    * If you do not yet have RHEL for s390x, go to the Red Hat [Customer Portal](https://access.redhat.com/downloads/content) and download it.
+## File Server
+* A file server accessible from your IBM zSystems / LinuxONE server.
+* Either FTP or HTTP service configured and active.
+* Once a RHEL server is installed natively on the LPAR, pre-existing or configured by this automation, (i.e. the KVM host), you can use that as the file server. 
+* If you are not using a pre-existing KVM host(s) and need to create them using this automation, you must use an FTP server because the HMC does not support HTTP.
+* A user with sudo and SSH access on that server.
+* A DVD ISO file of Red Hat Enterprise Linux (RHEL) 8 for s390x architecture mounted in an accessible folder (e.g. /home/<user>/rhel/ for FTP or /var/www/html/rhel for HTTP)
+    * If you do not have RHEL for s390x yet, go to the Red Hat [Customer Portal](https://access.redhat.com/downloads/content) and download it.
     * Under 'Product Variant' use the drop-down menu to select 'Red Hat Enterprise Linux for IBM z Systems' 
     * Double-check it's for version 8 and for s390x architecture
     * Then scroll down to Red Hat Enterprise Linux 8.x Binary DVD and click on the 'Download Now' button.
-* A folder to store config files (e.g. /home/ftpuser/ocp-config)
-* A user with sudo and SSH access.
+    * To pull the image directly from the command-line of your file server, copy the link for the 'Download Now' button and use `wget` to pull it down.
+        ```
+        wget "https://access.cdn.redhat.com/content/origin/files/sha256/13/13[...]40/rhel-8.7-s390x-dvd.iso?user=6[...]e"
+        ```
+    * Don't forget to mount it too:
+        * FTP:
+            ```
+            mount <rhel-8.7-s390x-dvd.iso> /home/<user>/rhel
+            ```
+        * or HTTP:
+            ```
+            mount <rhel-8.7-s390x-dvd.iso> /var/www/html/rhel
+            ```
+* A folder created to store config files (e.g. /home/user/ocp-config for FTP or /var/www/html/ocp-config for http)
+    * For FTP:
+        ```
+        sudo mkdir /home/<username>/ocp-config
+        ```
+        or HTTP:
+        ```
+        sudo mkdir /var/www/html/ocp-config
+        ```
 ## Ansible Controller
 * The computer/virtual machine running Ansible, sometimes referred to as localhost.
 * Must be running on with MacOS or Linux operating systems.
 * Network access to your IBM zSystems / LinuxONE hardware
 * All you need to run Ansible is a terminal and a text editor. However, an IDE like [VS Code](https://code.visualstudio.com/download) is highly recommended for an integrated, user-friendly experience with helpful extensions like [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml).
 * [Python3](https://realpython.com/installing-python/) installed:
-```
-brew install python3 #MacOS
-```
-```
-sudo dnf install python3 #Fedora
-```
-```
-sudo apt install python3 #Debian
-```
+    * MacOS, first install [Homebrew](https://brew.sh/) package manager:
+        ```
+        /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
+        ```
+        then install Python3
+        ```
+        brew install python3 #MacOS
+        ```
+    * Fedora:
+        ```
+        sudo dnf install python3 #Fedora
+        ```
+    * Debian:
+        ```
+        sudo apt install python3 #Debian
+        ```
 * Once Python3 is installed, you also need [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) version 2.9 or above:
 ```
 pip3 install ansible
 ```
-* Once Ansible is installed, you will need a few collections from Ansible Galaxy. Use this template to install them via the terminal:
+* Once Ansible is installed, you will need a few collections from Ansible Galaxy:
 ```
 ansible-galaxy collection install community.general community.crypto ansible.posix community.libvirt
 ```
 * If you will be using these playbooks to automate the creation of the LPAR(s) that will act as KVM host(s) for the cluster, you will also need:
 ```
 ansible-galaxy collection install ibm.ibm_zhmc 
 ```
-* If you are using MacOS, you also need to have [Homebrew](https://brew.sh/) package manager installed:
-```
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
-```
-  * [Xcode](https://apps.apple.com/us/app/xcode/id497799835?mt=12):
+* If you are using MacOS, you also need to have [Xcode](https://apps.apple.com/us/app/xcode/id497799835?mt=12):
 ```
 xcode-select --install
 ```

docs/run-the-playbooks.md

@@ -34,7 +34,7 @@ First-time setup of the Ansible Controller, the machine running Ansible.
 * Ansible inventory is templated out and working properly.
 * SSH key generated for Ansible passwordless authentication.
 * SSH agent is setup on the Ansible Controller.
-* Ansible SSH key is copied to the FTP server.
+* Ansible SSH key is copied to the file server.
 ### Notes
 * You can use an existing SSH key as your Ansible key, or have Ansible create one for you. It is highly recommended to use one without a passphrase.
 ## 1 Create LPAR Playbook
@@ -49,10 +49,10 @@ Creation of one to three Logical Partitions (LPARs), depending on your configura
 * Recommend opening the HMC via web-browser to watch the LPARs come up.
 ## 2 Create KVM Host Playbook
 ### Overview
-First-time start-up of Red Hat Enterprise Linux installed natively on the LPAR(s). Uses the Hardware Management Console (HMC) API, so your system must be in Dynamic Partition Manager (DPM) mode. Configuration files are passed to the FTP server and RHEL is booted and then kickstarted for fully automated setup.
+First-time start-up of Red Hat Enterprise Linux installed natively on the LPAR(s). Uses the Hardware Management Console (HMC) API, so your system must be in Dynamic Partition Manager (DPM) mode. Configuration files are passed to the file server and RHEL is booted and then kickstarted for fully automated setup.
 ### Outcomes
 * LPAR(s) started up in 'Active' state.
-* Configuration files (cfg, ins, prm) for the KVM host(s) are on the FTP server in the provided configs directory.
+* Configuration files (cfg, ins, prm) for the KVM host(s) are on the file server in the provided configs directory.
 ### Notes
 * Recommended to open the HMC via web-browser to watch the Operating System Messages for each LPAR as they boot in order to debug any potential problems.
 ## 3 Setup KVM Host Playbook
@@ -72,8 +72,7 @@ Configures the RHEL server(s) installed natively on the LPAR(s) to act as virtua
 ### Overview
 Creates the bastion KVM guest on the first KVM host. The bastion hosts essential services for the cluster. If you already have a bastion server, that can be used instead of running this playbook.
 ### Outcomes
-* RHEL ISO is mounted to HTTP-accessible directory on the FTP server.
-* Bastion configs are templated out to the FTP server.
+* Bastion configs are templated out to the file server.
 * Bastion is booted using virt-install.
 * Bastion is kickstarted for fully automated setup of the operating system.
 ### Notes

docs/set-variables-group-vars.md

@@ -33,14 +33,15 @@
 **env.z.lpar3.user** | Username for Linux admin on KVM host 3. Recommended to run as a non-root user with sudo access. | admin
 **env.z.lpar3.pass** | <b>(Optional)</b> The password for the admin user on the third KVM host. | ch4ngeMe!
 
-## 3 - FTP Server
+## 3 - File Server
 **Variable Name** | **Description** | **Example**
 :--- | :--- | :---
-**env.ftp.ip** | IPv4 address for the FTP server that will be used to pass config files and<br /> iso to KVM host LPAR(s) and bastion VM during their first boot. | 192.168.10.201
-**env.ftp.user** | Username to connect to the FTP server. Must have sudo and SSH access. | ftp-user
-**env.ftp.pass** | Password to connect to the FTP server as above user. | FTPpa$s!
-**env.ftp.iso_mount_dir** | Directory path relative to FTP root where RHEL ISO is mounted. If FTP root is /var/ftp/pub<br /> and the ISO is mounted at /var/ftp/pub/RHEL/8.5 then this variable would be<br /> RHEL/8.5. No slash before or after. | RHEL/8.5
-**env.ftp.cfgs_dir** | Directory path relative to FTP root where configuration files can be stored. If FTP root is /var/ftp/pub<br /> and you would like to store the configs at /var/ftp/pub/ocpz-config then this variable would be<br /> ocpz-config. No slash before or after. | ocpz-config
+**env.file_server.ip** | IPv4 address for the file server that will be used to pass config files and<br /> iso to KVM host LPAR(s) and bastion VM during their first boot. | 192.168.10.201
+**env.file_server.user** | Username to connect to the file server. Must have sudo and SSH access. | user1
+**env.file_server.pass** | Password to connect to the file server as above user. | user1pa$s!
+**env.file_server.protocol** | Protocol used to serve the files, either 'ftp' or 'http' | http
+**env.file_server.iso_mount_dir** | Directory path relative to the HTTP/FTP accessible directory where RHEL ISO is mounted. For example, if the FTP root is at /home/user1<br /> and the ISO is mounted at /home/user1/RHEL/8.7 then this variable would be<br /> RHEL/8.7 - no slash before or after. | RHEL/8.7
+**env.file_server.cfgs_dir** | Directory path relative to to the HTTP/FTP accessible directory where configuration files can be stored. For example, if FTP root is /home/user1<br /> and you would like to store the configs at /home/user1/ocpz-config then this variable would be<br /> ocpz-config. No slash before or after. | ocpz-config
 
 ## 4 - Red Hat Info
 **Variable Name** | **Description** | **Example**

inventories/default/group_vars/all.yaml.template

@@ -31,11 +31,12 @@ env:
 #      user:
 #      pass:
 
-# Section 3 - FTP Server
-  ftp:
+# Section 3 - File Server
+  file_server:
     ip: #X
     user: #X
     pass: #X
+    protocol: #X
     iso_mount_dir: #X
     cfgs_dir: #X
 

playbooks/0_setup.yaml

@@ -59,9 +59,9 @@
     - "{{ inventory_dir }}/group_vars/all.yaml"
   vars:
     packages: "{{ env.pkgs.controller }}"
-    ssh_target: [ "{{ env.ftp.ip }}", "{{ env.ftp.user }}", "{{ env.ftp.pass }}", "{{ path_to_key_pair }}" ]
+    ssh_target: [ "{{ env.file_server.ip }}", "{{ env.file_server.user }}", "{{ env.file_server.pass }}", "{{ path_to_key_pair }}" ]
   roles:
     - install_packages
     - ssh_key_gen
     - ssh_agent
-    - ssh_copy_id #to FTP server
+    - ssh_copy_id #to file server

playbooks/2_create_kvm_host.yaml

@@ -1,6 +1,6 @@
 ---
 #Template and update RHEL configuration files for KVM host to boot from
-- hosts: ftp
+- hosts: file_server
   tags: update_cfgs, update, kvm_host_1
   become: false
   gather_facts: false
@@ -18,7 +18,7 @@
       when: env.z.lpar1.create == True
 
 #Template and update RHEL configuration files for KVM host to boot from
-- hosts: ftp
+- hosts: file_server
   tags: update_cfgs, update, kvm_host_2
   become: false
   gather_facts: false
@@ -36,7 +36,7 @@
       when: env.z.lpar2.create == True
 
 #Template and update RHEL configuration files for KVM host to boot from
-- hosts: ftp
+- hosts: file_server
   tags: update_cfgs, update, kvm_host_3
   become: false
   gather_facts: false

roles/create_bastion/tasks/main.yaml

@@ -4,7 +4,7 @@
   become: false
   tags: create_bastion
   ansible.builtin.file:
-    path: "{{ env.ftp.cfgs_dir }}/{{ env.bastion.networking.hostname }}"
+    path: "{{ env.file_server.cfgs_dir }}/{{ env.bastion.networking.hostname }}"
     mode: "0755"
     state: directory
 
@@ -13,7 +13,7 @@
   become: false
   ansible.builtin.template:
     src: "bastion-ks.cfg.j2"
-    dest: "{{ env.ftp.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
+    dest: "{{ env.file_server.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
     mode: "0644"
     force: true
 
@@ -28,7 +28,7 @@
   tags: create_bastion
   become: false
   ansible.builtin.lineinfile:
-    path: "{{ env.ftp.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
+    path: "{{ env.file_server.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
     insertafter: "Root password"
     line: "rootpw --iscrypted {{ root_pass_hash.stdout }}"
 
@@ -43,7 +43,7 @@
   tags: create_bastion
   become: false
   ansible.builtin.lineinfile:
-    path: "{{ env.ftp.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
+    path: "{{ env.file_server.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg"
     insertafter: "Users and Groups Definitions"
     line: "user --groups=wheel --name={{ env.bastion.access.user }} --password={{ user_pass_hash.stdout }} --iscrypted"
 
@@ -64,11 +64,11 @@
     --autostart \
     --memory={{ env.bastion.resources.ram }} \
     --vcpus={{ env.bastion.resources.vcpu }} \
-    --location http://{{ env.ftp.ip }}/{{ env.ftp.iso_mount_dir }} \
+    --location http://{{ env.file_server.ip }}/{{ env.file_server.iso_mount_dir }} \
     --disk pool={{ env.cluster.networking.metadata_name }}-vdisk,size={{ env.bastion.resources.disk_size }}  \
     --network network={{ env.bridge_name }} \
     --graphics=none \
     --noautoconsole --wait=-1 \
-    --initrd-inject "/{{ kvm_host_home.stdout }}/{{ env.ftp.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg" \
+    --initrd-inject "/{{ kvm_host_home.stdout }}/{{ env.file_server.cfgs_dir }}/{{ env.bastion.networking.hostname }}/bastion-ks.cfg" \
     --extra-args "inst.ks=file:/bastion-ks.cfg ip={{ env.bastion.networking.ip }}::{{ env.bastion.networking.gateway }}\
     :{{ env.bastion.networking.subnetmask }}:{{ env.bastion.networking.hostname }}:enc1:none console=ttysclp0"

roles/create_bastion/templates/bastion-ks.cfg.j2

@@ -14,11 +14,11 @@ text --non-interactive
 firstboot --enable
 
 # Use network installation
-url --url=http://{{ env.ftp.ip }}/{{ env.ftp.iso_mount_dir }}
+url --url={{ env.file_server.protocol }}://{{ env.file_server.user + ':' + env.file_server.pass + '@' if env.file_server.protocol == 'ftp' else '' }}{{ env.file_server.ip }}/{{ env.file_server.iso_mount_dir }}
 
 # Add yum repositories
-repo --install --name="AppStream" --baseurl=http://{{ env.ftp.ip }}/{{ env.ftp.iso_mount_dir }}/AppStream/
-repo --install --name="BaseOS" --baseurl=http://{{ env.ftp.ip }}/{{ env.ftp.iso_mount_dir }}/BaseOS/
+repo --install --name="AppStream" --baseurl={{ env.file_server.protocol }}://{{ env.file_server.user + ':' + env.file_server.pass + '@' if env.file_server.protocol == 'ftp' else '' }}{{ env.file_server.ip }}/{{ env.file_server.iso_mount_dir }}/AppStream/
+repo --install --name="BaseOS" --baseurl={{ env.file_server.protocol }}://{{ env.file_server.user + ':' + env.file_server.pass + '@' if env.file_server.protocol == 'ftp' else '' }}{{ env.file_server.ip }}/{{ env.file_server.iso_mount_dir }}/BaseOS/
 
 # Keyboard layouts
 keyboard --vckeymap=us --xlayouts='us'

roles/create_kvm_host/tasks/main.yaml

@@ -11,10 +11,10 @@
     cpc_name: "{{ cpc_name }}"
     name: "{{ lpar.name }}"
     properties:
-      boot_ftp_host: "{{ env.ftp.ip }}"
-      boot_ftp_username: "{{ env.ftp.user }}"
-      boot_ftp_password: "{{ env.ftp.pass }}"
-      boot_ftp_insfile: "{{ env.ftp.cfgs_dir }}/{{ lpar.name }}/kvm_host.ins"
+      boot_ftp_host: "{{ env.file_server.ip }}"
+      boot_ftp_username: "{{ env.file_server.user }}"
+      boot_ftp_password: "{{ env.file_server.pass }}"
+      boot_ftp_insfile: "{{ env.file_server.cfgs_dir }}/{{ lpar.name }}/kvm_host.ins"
       boot_device: "ftp"
     state: active
   register: _create_instances

roles/set_inventory/tasks/main.yaml

@@ -19,7 +19,7 @@
     msg: "Error jumphost vars undefined: when env.network_mode is NAT, you must set all env.jumphost variables."
   when: ( env.network_mode | upper == 'NAT' ) and (env.jumphost.name is none or env.jumphost.ip is none or env.jumphost.user is none or env.jumphost.pass is none or env.jumphost.path_to_keypair is none)
 
-- name: Template out inventory with localhost, FTP, KVM host, jumphost(optional) and bastion information
+- name: Template out inventory with localhost, file server, KVM host, jumphost(optional) and bastion information
   tags: set_inventory
   template:
     src: hosts.j2

roles/set_inventory/templates/hosts.j2

@@ -1,8 +1,8 @@
 [localhost]
 127.0.0.1 ansible_connection=local ansible_become_password={{ env.controller.sudo_pass }}
 
-[ftp]
-{{ env.ftp.ip }} ansible_user={{ env.ftp.user }} ansible_become_password={{ env.ftp.pass }}
+[file_server]
+{{ env.file_server.ip }} ansible_user={{ env.file_server.user }} ansible_become_password={{ env.file_server.pass }}
 
 [kvm_host]
 {{ env.z.lpar1.hostname }} ansible_host={{ env.z.lpar1.ip }} ansible_user={{ env.z.lpar1.user }} ansible_become_password={{ env.z.lpar1.pass }}