PVE Launch Cluster

Launch a new PVE Cluster

Directions for creating a lab PVE cluster with three nodes named gadget1, gadget2 and gadget3.

This assumes a working and configured Ansible Control Node and that all prerequisites have been met.

Steps:

1. Install PVE
2. Update inventory
3. Acquire Images
4. Create SSH key
5. Configure SSH access
6. Install Ansible Collections
7. Run playbook

Objective:

• Configure APT repositories for No-Subscription and update PVE nodes.
• Configure authorized_keys for root to include (and remove) desired SSH public keys.
• Configure the nodes into a PVE cluster named lab.
• Upload Cloud-init image files to PVE nodes.

Showcased roles

• cloudcodger.proxmox_ve.apt
• cloudcodger.proxmox_ve.authorized_keys
• cloudcodger.proxmox_ve.cluster
• cloudcodger.proxmox_ve.upload
• cloudcodger.proxmox_openssh.datacenter

The links to Ansible Collections.

Regarding the cloudcodger.proxmox_ve.upload role variables. These have been configured to lookup the files that exist with the assumption that all Cloud-Init image end with .qcow2, all ISO images end in .iso and all LXC templates end with .zst. If you would like to change what gets uploaded, either edit the lab-minimal/group_vars/proxmox_hosts.yml file to set specific lists or update the contents of the files directory before running the playbook. Keep in mind that this does not remove any files on the PVE nodes and only uploads new or changed files.

Install PVE

First install the PVE software on the machines that will be your lab environment. For experimentation purposes, these could be VMs. Keep track of the root password set during the installation process as this will be needed when configuring SSH access.

The showcase uses the hostnames of gadget1, gadget2 and gadget3 for the three PVE nodes.

Update Inventory

Make sure the IP addresses in lab-minimal/pve.yml are correct for the PVE nodes.

Acquire Images

Download any cloud-init images, ISO images and LXC container templates you want uploaded to the PVE node during the Run playbook step. These are listed in the upload_cloud_init_files, upload_iso_images and upload_lxc_templates variables respectively. To simplify things, these are set in the lab-minimal/group_vars/proxmox_hosts.yml file, where it looks up the existing files by the extention for each of the types.

The acquire.yml playbook will download the images and templates listed in the loop values for each task. Update the playbook with the desired items before running it.

This step can be skipped if the desired files already exist in the files directory. Make sure you have any files to be uploaded by the cloudcodger.proxmox_ve.upload role. This playbook runs on localhost, which does not require an inventory. However, this will produce the messages

[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Providing an inventory file with -i prevents these warning messages. The local_aliases.yml from any of the Inventory directories can be used for this command.

ansible-playbook -i basic/local_aliases.yml acquire.yml

ansible-playbook -i lab/local_aliases.yml acquire.yml

Create SSH key

If an ed25519 type SSH key hasn't been created yet, create one. The file name needs to match the one in lab-minimal/group_vars/proxmox_hosts.yml. Otherwise the lookup will fail.

ssh-keygen -t ed25519 -N "" -f ~/.ssh/showcase_ed25519

The -N sets an empty passphrase which is OK for a lab and using one is left up to the reader.

Configure SSH access

The known_hosts.yml playbook, with the specific inventory file lab-minimal/pve.yml, will scan each host and add the SSH hostkey to ~/.ssh/known_hosts file on the Ansible Control Node.

ansible-playbook -i lab-minimal/pve.yml known_hosts.yml -e group_name=proxmox_hosts

Set up passwordless SSH access to the PVE nodes by configuring an SSH key pair for the root user. The recommended method of doing this is to use ssh-copy-id. Substitue IP addresses if name resolution isn't configured.

ssh-copy-id -i ~/.ssh/showcase.pub root@gadget1

ssh-copy-id -i ~/.ssh/showcase.pub root@gadget2

ssh-copy-id -i ~/.ssh/showcase.pub root@gadget3

Alternatively, thepve_authorized_keys.yml playbook, with the specific inventory file lab-minimal/pve.yml, will add the configured SSH keys. The -k will cause Ansible to prompt for the password that was set during the install. This option was not mentioned in the PVE Launch Node page because of a pmxcfs limitation.

ansible-playbook -k -i lab-minimal/pve.yml pve_authorized_keys.yml

This entire step could also be skipped by adding -k --ssh-common-args='-o StrictHostKeyChecking=accept-new' to the command when running the playbook. This option was also not mentioned in the PVE Launch Node page because of the pmxcfs limitation. As well as the fact that pve_node.yml doesn't call the cloudcodger.proxmox_ve.authorized_keys role, for the same reason, and would then require the -k option with every execution of ansible-playbook until you set up passwordless SSH access.

Ansible Collections

The pve.yml playbook requires multiple collections and roles. If not done as part of creating the Ansible Control Node, install them using the requirements.yml file.

ansible-galaxy collection install -r requirements.yml

Run playbook

Run the pve_cluster.yml Ansible playbook with the specific inventory file lab-minimal/pve.yml. This must be able to validate certificates on the PVE nodes. The PVE Certificates page describes one method of configuring them.

ansible-playbook -i lab-minimal/pve.yml pve_cluster.yml

Playbook Contents

This playbook demonstrates the showcased roles in two plays. Described here.

The configure hosts play

The first play runs on each PVE node in the proxmox_hosts group (set in lab-minimal/pve.yml) with settings in lab-minimal/group_vars/proxmox_hosts.yml.

Roles executed:

• cloudcodger.proxmox_ve.apt to:

  • update the APT repositories
  • run apt dist-upgrade
  • reboot the PVE node if needed

• cloudcodger.proxmox_ve.cluster to:

  • create the PVE cluster
  • join other PVE nodes to the cluster

• cloudcodger.proxmox_ve.authorized_keys to:

  • update ~/.ssh/authorized_keys file for the root user
  • runs after the cluster role because of the pmxcfs limitation

• cloudcodger.proxmox_ve.upload to:

  • upload cloud-init images
  • upload ISO images
  • upload template images

Configure Datacenter play

The second play runs on local_pve_datacenter (an alias for localhost in lab-minimal/local_aliases.yml) with settings in lab-minimal/host_vars/local_pve_datacenter.yml.

Role executed:

• The cloudcodger.proxmox_openssh.datacenter role will:
  • Create a .pve_secrets directory on the control node.
  • Update the local storage allowed content, so images is included.
  • Create Administrator and Auditor groups, users, tokens, and permission ACLs. Storing token secrets in the .pve_secrets directory.