Skip to content

Commit

Permalink
Adding virtualbox local build for local environment base box
Browse files Browse the repository at this point in the history
  • Loading branch information
knabben committed Sep 21, 2021
1 parent 6eacd34 commit 5faad97
Show file tree
Hide file tree
Showing 16 changed files with 575 additions and 5 deletions.
1 change: 1 addition & 0 deletions docs/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [OpenStack](./capi/providers/openstack.md)
- [raw](./capi/providers/raw.md)
- [vSphere](./capi/providers/vsphere.md)
- [VirtualBox](./capi/providers/virtualbox.md)
- [Testing the Images](./capi/goss/goss.md)
- [Using Container Images](./capi/container-image.md)
- [Glossary](./glossary.md)
1 change: 1 addition & 0 deletions docs/book/src/capi/capi.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ If any needed binaries are not present, they can be installed to `images/capi/.b
* [OpenStack](./providers/openstack.md)
* [Raw](./providers/raw.md)
* [vSphere](./providers/vsphere.md)
* [VirtualBox](./providers/virtualbox.md)

## Make targets

Expand Down
31 changes: 31 additions & 0 deletions docs/book/src/capi/providers/virtualbox.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Building Images for VirtualBox

## Hypervisor

The image is built using Oracle VM VirtualBox hypervisor.

### Installing VirtualBox package

Oracle VirtualBox install instructions and packages are available at the [official page](https://www.virtualbox.org/wiki/Downloads).

## Building Images

### Validating

The build [prerequisites](../capi.md#prerequisites) for using `image-builder` for
building vbox images are managed by running:

```bash
cd image-builder/images/capi
make deps-vbox
```

### Generating a VirtualBox image

Only Windows 2019 images are available for VirtualBox hypervisor for now, to build local images
for development are made by running:

```bash
cd image-builder/images/capi
make build-node-vbox-local-windows-2019
```
4 changes: 4 additions & 0 deletions docs/book/src/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ An open standard for packaging and distributing virtual appliances or, more gene

vCenter can be thought of as the management layer for ESXi hosts. Hosts can be arranged into Datacenters, Clusters or resources pools, vCenter is the centralized monitoring and management control plane for ESXi hosts allow centralized management, integration points for other products in the VMware SDDC stack and third party solutions, like backup, DR or networking overlay applications, such as NSX. vCenter also provides all of the higher level features of vSphere such as vMotion, vSAN, HA, DRS, Distributed Switches and more.

## VirtualBox

VirtualBox is a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use.

## VM

A VM is an abstraction of an operating system from the physical machine by creating a "virtual" representation of the physical hardware the OS expects to interact with, this includes but is not limited to CPU instruction sets, memory, BIOS, PCI buses, etc. A VM is an entirely self-contained entity and shares no components with the host OS. In the case of vSphere the host OS is ESXi (see below).
Expand Down
44 changes: 39 additions & 5 deletions images/capi/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2019 The Kubernetes Authors.
# Copyright 2021 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -47,7 +47,7 @@ version: ## Display version of image-builder

.PHONY: deps
deps: ## Installs/checks all dependencies
deps: deps-ami deps-azure deps-do deps-gce deps-ova deps-qemu deps-raw
deps: deps-ami deps-azure deps-do deps-gce deps-ova deps-qemu deps-raw deps-vbox

.PHONY: deps-ami
deps-ami: ## Installs/checks dependencies for AMI builds
Expand All @@ -67,7 +67,6 @@ deps-azure:
hack/ensure-azure-cli.sh
hack/ensure-goss.sh


.PHONY: deps-do
deps-do: ## Installs/checks dependencies for DigitalOcean builds
deps-do:
Expand Down Expand Up @@ -101,6 +100,15 @@ deps-raw:
hack/ensure-ansible.sh
hack/ensure-packer.sh

.PHONY: deps-vbox
deps-vbox: ## Installs/checks dependencies for VirtualBox builds
deps-vbox:
hack/ensure-ansible.sh
hack/ensure-ansible-windows.sh
hack/ensure-packer.sh
hack/ensure-goss.sh


## --------------------------------------
## Container variables
## --------------------------------------
Expand Down Expand Up @@ -235,6 +243,7 @@ NODE_OVA_VSPHERE_CLONE_BUILD_NAMES := $(addprefix node-ova-vsphere-clone-,$(PLA
HAPROXY_OVA_LOCAL_BUILD_NAMES := $(addprefix haproxy-ova-local-,$(PHOTON_VERSIONS))
HAPROXY_OVA_ESX_BUILD_NAMES := $(addprefix haproxy-ova-esx-,$(PHOTON_VERSIONS))
HAPROXY_OVA_VSPHERE_BUILD_NAMES := $(addprefix haproxy-ova-vsphere-,$(PHOTON_VERSIONS))
NODE_VBOX_LOCAL_BUILD_NAMES := $(addprefix node-vbox-local-,$(WINDOWS_VERSIONS))

AMI_BUILD_NAMES ?= ami-centos-7 ami-ubuntu-1804 ami-ubuntu-2004 ami-amazon-2 ami-flatcar ami-windows-2019 ami-windows-2004
GCE_BUILD_NAMES ?= gce-ubuntu-1804 ## gce-ubuntu-2004
Expand Down Expand Up @@ -282,6 +291,9 @@ QEMU_BUILD_TARGETS := $(addprefix build-,$(QEMU_BUILD_NAMES))
QEMU_VALIDATE_TARGETS := $(addprefix validate-,$(QEMU_BUILD_NAMES))
RAW_BUILD_TARGETS := $(addprefix build-,$(RAW_BUILD_NAMES))
RAW_VALIDATE_TARGETS := $(addprefix validate-,$(RAW_BUILD_NAMES))
NODE_VBOX_BUILD_TARGETS := $(addprefix build-,$(NODE_VBOX_LOCAL_BUILD_NAMES))
NODE_VBOX_VALIDATE_TARGETS := $(addprefix validate-,$(NODE_VBOX_LOCAL_BUILD_NAMES))


.PHONY: $(NODE_OVA_LOCAL_BUILD_TARGETS)
$(NODE_OVA_LOCAL_BUILD_TARGETS): deps-ova
Expand All @@ -307,7 +319,6 @@ $(NODE_OVA_ESX_BUILD_TARGETS): deps-ova
$(NODE_OVA_VSPHERE_BUILD_TARGETS): deps-ova
packer build $(if $(findstring windows,$@),$(PACKER_WINDOWS_NODE_FLAGS),$(PACKER_NODE_FLAGS)) -var-file="packer/ova/packer-common.json" -var-file="$(abspath packer/ova/$(subst build-node-ova-vsphere-,,$@).json)" -var-file="packer/ova/vsphere.json" -except=esx -except=local -only=vsphere-iso $(ABSOLUTE_PACKER_VAR_FILES) -only=vsphere packer/ova/packer-$(if $(findstring windows,$@),windows,node).json


.PHONY: $(NODE_OVA_VSPHERE_BASE_BUILD_TARGETS)
$(NODE_OVA_VSPHERE_BASE_BUILD_TARGETS): deps-ova
packer build $(PACKER_NODE_FLAGS) -var-file="packer/ova/packer-common.json" -var-file="$(abspath packer/ova/$(subst build-node-ova-vsphere-base-,,$@).json)" -var-file="packer/ova/vsphere.json" -except=esx -except=local -except=manifest -except=vsphere -only=vsphere-iso-base $(ABSOLUTE_PACKER_VAR_FILES) packer/ova/packer-node.json
Expand Down Expand Up @@ -404,6 +415,14 @@ $(RAW_BUILD_TARGETS): deps-raw
$(RAW_VALIDATE_TARGETS): deps-raw
packer validate $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/raw/$(subst validate-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) -except=flatcar packer/raw/packer.json

.PHONY: $(NODE_VBOX_BUILD_TARGETS)
$(NODE_VBOX_BUILD_TARGETS): deps-vbox
packer build $(if $(findstring windows,$@),$(PACKER_WINDOWS_NODE_FLAGS),$(PACKER_NODE_FLAGS)) -var-file="packer/vbox/packer-common.json" -var-file="$(abspath packer/vbox/$(subst build-node-vbox-local-,,$@).json)" -except=esx -except=vsphere -only=virtualbox-iso $(ABSOLUTE_PACKER_VAR_FILES) packer/vbox/packer-$(if $(findstring windows,$@),windows).json

.PHONY: $(NODE_VBOX_VALIDATE_TARGETS)
$(NODE_VBOX_VALIDATE_TARGETS): deps-vbox
packer validate $(if $(findstring windows,$@),$(PACKER_WINDOWS_NODE_FLAGS),$(PACKER_NODE_FLAGS)) -var-file="packer/ova/packer-common.json" -var-file="$(abspath packer/vbox/$(subst validate-node-vbox-local-,,$@).json)" -except=esx -except=vsphere -only=virtualbox-iso $(ABSOLUTE_PACKER_VAR_FILES) packer/vbox/packer-$(if $(findstring windows,$@),windows).json


## --------------------------------------
## Dynamic clean targets
Expand All @@ -423,11 +442,16 @@ QEMU_CLEAN_TARGETS := $(subst build-,clean-,$(QEMU_BUILD_TARGETS))
$(QEMU_CLEAN_TARGETS):
rm -fr output/$(subst clean-qemu-,,$@)-kube*

RAW_CLEAN_TARGETS := $(subst build-,clean-,$(RAW_CLEAN_TARGETS))
RAW_CLEAN_TARGETS := $(subst build-,clean-,$(RAW_BUILD_TARGETS))
.PHONY: $(RAW_CLEAN_TARGETS)
$(RAW_CLEAN_TARGETS):
rm -fr output/$(subst clean-raw-,,$@)-kube*

VBOX_CLEAN_TARGETS := $(subst build-,clean-,$(NODE_VBOX_BUILD_TARGETS))
.PHONY: $(VBOX_CLEAN_TARGETS)
$(VBOX_CLEAN_TARGETS):
rm -fr output/$(subst clean-node-vbox-local-,,$@)-kube*

## --------------------------------------
## Document dynamic build targets
## --------------------------------------
Expand Down Expand Up @@ -536,6 +560,9 @@ build-raw-ubuntu-1804: ## Builds Ubuntu 18.04 RAW image
build-raw-ubuntu-2004: ## Builds Ubuntu 20.04 RAW image
build-raw-all: $(RAW_BUILD_TARGETS) ## Builds all RAW images

build-node-vbox-local-windows-2019: ## Builds for Windows Server 2019 Node VirtualBox w local hypervisor


## --------------------------------------
## Document dynamic validate targets
## --------------------------------------
Expand Down Expand Up @@ -605,6 +632,8 @@ validate-raw-ubuntu-1804: ## Validates Ubuntu 18.04 RAW image packer config
validate-raw-ubuntu-2004: ## Validates Ubuntu 20.04 RAW image packer config
validate-raw-all: $(RAW_VALIDATE_TARGETS) ## Validates all RAW Packer config

validate-node-vbox-local-windows-2019: ## Validates Windows Server 2019 Node VirtualBox Packer config w local hypervisor

validate-all: validate-ami-all \
validate-azure-all \
validate-do-all \
Expand Down Expand Up @@ -636,6 +665,11 @@ clean-qemu: $(QEMU_CLEAN_TARGETS)
clean-raw: ## Removes all raw image output directories (see NOTE at top of help)
clean-raw: $(RAW_CLEAN_TARGETS)

.PHONY: clean-vbox
clean-vbox: ## Removes all vbox image output directories (see NOTE at top of help)
clean-vbox: $(VBOX_CLEAN_TARGETS)


.PHONY: clean-packer-cache
clean-packer-cache: ## Removes the packer cache
clean-packer-cache:
Expand Down
4 changes: 4 additions & 0 deletions images/capi/packer/vbox/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# See the OWNERS docs at https://go.k8s.io/owners

approvers:
- cluster-api-vsphere-maintainers
33 changes: 33 additions & 0 deletions images/capi/packer/vbox/packer-common.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"boot_wait": "10s",
"convert_to_template": "false",
"cpu": "4",
"cpu_cores": "1",
"disk_controller_type": "pvscsi",
"disk_thin_provisioned": "true",
"disk_type_id": "0",
"firmware": "bios",
"format": "",
"guestinfo_datasource_ref": "v1.4.0",
"guestinfo_datasource_script": "{{user `guestinfo_datasource_slug`}}/{{user `guestinfo_datasource_ref`}}/install.sh",
"guestinfo_datasource_slug": "https://raw.githubusercontent.com/vmware/cloud-init-vmware-guestinfo",
"headless": "true",
"insecure_connection": "false",
"memory": "8192",
"network": "",
"network_card": "vmxnet3",
"remote_datastore": "",
"remote_host": "",
"remote_password": "",
"remote_type": "",
"remote_username": "",
"skip_compaction": "false",
"ssh_password": "builder",
"ssh_timeout": "60m",
"ssh_username": "builder",
"vmx_version": "15",
"vnc_bind_address": "127.0.0.1",
"vnc_disable_password": "false",
"vnc_port_max": "6000",
"vnc_port_min": "5900"
}
130 changes: 130 additions & 0 deletions images/capi/packer/vbox/packer-windows.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
{
"builders": [
{
"boot_wait": "{{user `boot_wait`}}",
"communicator": "winrm",
"cpus": "{{user `cpu`}}",
"disk_size": "{{user `disk_size`}}",
"floppy_files": [
"./packer/vbox/windows/{{user `build_name`}}/autounattend.xml",
"./packer/vbox/windows/disable-network-discovery.cmd",
"./packer/vbox/windows/disable-winrm.ps1",
"./packer/vbox/windows/enable-winrm.ps1",
"./packer/vbox/windows/sysprep.ps1"
],
"guest_additions_mode": "disable",
"guest_os_type": "{{user `local_guest_os_type`}}",
"iso_checksum": "{{user `iso_checksum`}}",
"iso_urls": [
"{{user `os_iso_url`}}"
],
"memory": "{{user `memory`}}",
"name": "virtualbox-iso",
"output_directory": "{{user `output_dir`}}",
"shutdown_command": "powershell A:/sysprep.ps1",
"shutdown_timeout": "1h",
"type": "virtualbox-iso",
"vm_name": "{{user `build_version`}}",
"winrm_password": "S3cr3t0!",
"winrm_timeout": "4h",
"winrm_username": "Administrator"
}
],
"post-processors": [
{
"keep_input_artifact": true,
"output": "./output/windows-2019.box",
"type": "vagrant",
"vagrantfile_template": "./packer/vbox/vagrantfile-windows_2019.template"
}
],
"provisioners": [
{
"extra_arguments": [
"-e",
"ansible_winrm_scheme=http",
"--extra-vars",
"{{user `ansible_common_vars`}}",
"--extra-vars",
"{{user `ansible_extra_vars`}}",
"--extra-vars",
"{{user `ansible_user_vars`}}"
],
"playbook_file": "ansible/windows/node_windows.yml",
"type": "ansible",
"use_proxy": false,
"user": "Administrator"
},
{
"restart_check_command": "powershell -command \"& {if ((get-content C:\\ProgramData\\lastboot.txt) -eq (Get-WmiObject win32_operatingsystem).LastBootUpTime) {Write-Output 'Sleeping for 600 seconds to wait for reboot'; start-sleep 600} else {Write-Output 'Reboot complete'}}\"",
"restart_command": "powershell \"& {(Get-WmiObject win32_operatingsystem).LastBootUpTime > C:\\ProgramData\\lastboot.txt; Restart-Computer -force}\"",
"type": "windows-restart"
},
{
"arch": "{{user `goss_arch`}}",
"download_path": "{{user `goss_download_path`}}",
"format": "{{user `goss_format`}}",
"format_options": "{{user `goss_format_options`}}",
"goss_file": "{{user `goss_entry_file`}}",
"inspect": "{{user `goss_inspect_mode`}}",
"remote_folder": "{{user `goss_remote_folder`}}",
"remote_path": "{{user `goss_remote_path`}}",
"skip_install": "{{user `goss_skip_install`}}",
"target_os": "Windows",
"tests": [
"{{user `goss_tests_dir`}}"
],
"type": "goss",
"url": "{{user `goss_url`}}",
"use_sudo": false,
"vars_env": {
"GOSS_MAX_CONCURRENT": "1",
"GOSS_USE_ALPHA": "1"
},
"vars_file": "{{user `goss_vars_file`}}",
"vars_inline": {
"OS": "{{user `distro_name` | lower}}",
"PROVIDER": "ova",
"containerd_version": "{{user `containerd_version`}}",
"distribution_version": "{{user `distro_version`}}",
"docker_ee_version": "{{user `docker_ee_version`}}",
"kubernetes_version": "{{user `kubernetes_semver`}}",
"pause_image": "{{user `pause_image`}}",
"runtime": "{{user `runtime`}}"
},
"version": "{{user `goss_version`}}"
}
],
"variables": {
"additional_debug_files": null,
"ansible_common_vars": "",
"ansible_extra_vars": "",
"ansible_user_vars": "",
"build_name": null,
"build_timestamp": "{{timestamp}}",
"build_version": "{{user `build_name`}}-kube-{{user `kubernetes_semver`}}",
"cloudbase_init_url": "https://github.com/cloudbase/cloudbase-init/releases/download/{{user `cloudbase_init_version`}}/CloudbaseInitSetup_{{user `cloudbase_init_version` | replace_all `.` `_` }}_x64.msi",
"cloudbase_metadata_services": "cloudbaseinit.metadata.services.vmwareguestinfoservice.VMwareGuestInfoService",
"cloudbase_metadata_services_unattend": "cloudbaseinit.metadata.services.vmwareguestinfoservice.VMwareGuestInfoService",
"cloudbase_plugins": "cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin, cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin, cloudbaseinit.plugins.common.userdata.UserDataPlugin, cloudbaseinit.plugins.common.localscripts.LocalScriptsPlugin, cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin",
"cloudbase_plugins_unattend": "cloudbaseinit.plugins.common.mtu.MTUPlugin",
"containerd_sha256": null,
"containerd_url": "",
"containerd_version": null,
"disable_hypervisor": null,
"disk_size": "81920",
"ib_version": "{{env `IB_VERSION`}}",
"kubernetes_base_url": "https://kubernetesreleases.blob.core.windows.net/kubernetes/{{user `kubernetes_semver`}}/binaries/node/windows/{{user `kubernetes_goarch`}}",
"kubernetes_http_package_url": "",
"kubernetes_typed_version": "kube-{{user `kubernetes_semver`}}",
"manifest_output": "manifest.json",
"netbios_host_name_compatibility": null,
"nssm_url": null,
"output_dir": "./output/{{user `build_version`}}",
"prepull": null,
"windows_service_manager": null,
"windows_updates_categories": null,
"windows_updates_kbs": null,
"wins_url": "https://github.com/rancher/wins/releases/download/v{{user `wins_version`}}/wins.exe"
}
}
28 changes: 28 additions & 0 deletions images/capi/packer/vbox/vagrantfile-windows_2019.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.require_version ">= 1.6.2"

Vagrant.configure("2") do |config|
config.vm.define "vagrant-windows-2019"
config.vm.box = "windows_2019"
config.vm.communicator = "winrm"

# Admin user name and password
config.winrm.username = "Administrator"
config.winrm.password = "S3cr3t0!"

config.vm.guest = :windows
config.windows.halt_timeout = 15

config.vm.network :forwarded_port, guest: 3389, host: 3389, id: "rdp", auto_correct: true

config.vm.provider :virtualbox do |v, override|
#v.gui = true
v.customize ["modifyvm", :id, "--memory", 2048]
v.customize ["modifyvm", :id, "--cpus", 2]
v.customize ["modifyvm", :id, "--vram", 128]
v.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
v.customize ["setextradata", "global", "GUI/SuppressMessages", "all" ]
end
end
11 changes: 11 additions & 0 deletions images/capi/packer/vbox/windows-2019.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"build_name": "windows-2019",
"distro_arch": "amd64",
"distro_name": "windows",
"distro_version": "2019",
"guest_os_type": "Windows2019Server-64",
"iso_checksum": "none",
"local_guest_os_type": "windows9srv-64",
"os_display_name": "Windows Server 2019",
"os_iso_url": "file:/path/en_windows_server_2019_x64_dvd_4cb967d8.iso"
}
Loading

0 comments on commit 5faad97

Please sign in to comment.