docs(virtualization): add a VMware vSphere migration guide - #673
Open
Andrei Kvapil (kvaps) wants to merge 4 commits into
Open
docs(virtualization): add a VMware vSphere migration guide#673Andrei Kvapil (kvaps) wants to merge 4 commits into
Andrei Kvapil (kvaps) wants to merge 4 commits into
Conversation
Documents the forklift.cozystack.io import API: registering a vCenter connection, running an import, and what the result is. Sits beside the Proxmox guide, which covers the manual export-and-upload path. The pre-flight section is the part that matters. Every check in it corresponds to a failure observed on a live cluster, and each one surfaces late and names something other than its cause: a WaitForFirstConsumer storage class that deadlocks the transfer, an ESXi address colliding with the cluster Service CIDR so the VDDK connection is swallowed by service routing, a vCenter username missing its SSO domain reported as a wrong password, and a Forklift deployment serving a certificate its own rotated CA no longer matches. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
The VDDK is licensed software: VMware distributes it to registered users only and nobody may redistribute it, which is why Cozystack ships no image and the platform value starts empty. That leaves an administrator with a build step and no instructions for it. Covers where to get the kit, matching its version to vSphere, the Containerfile, building for linux/amd64 (an arm64 build pulls and then fails to execute on the workers), making a private registry pullable from the namespaces that need it, and a one-liner to confirm the pull before the first migration rather than during it. Also notes that Proxmox is not a provider the engine supports, so readers arriving from Proxmox are sent to the guide that applies to them instead of working through a vCenter setup that will not help. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Leaves the page to describe the vSphere path on its own terms. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
✅ Deploy Preview for cozystack ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The limitations section said a guest must already carry virtio drivers, which stopped being true when imports started defaulting to the SATA bus. It also understated the failure: a guest without virtio does not fail the import, it imports successfully and then does not boot. Adds the section the lab run showed was missing — why the bus is what it is, how to move to virtio once the drivers are in place, and what happens to a machine with more disks than the controller has ports. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
11 tasks
Andrei Kvapil (kvaps)
marked this pull request as ready for review
September 7, 2026 13:24
Andrei Kvapil (kvaps)
requested review from
Timofei Larkin (lllamnyp),
myasnikovdaniil and
Timur Tukaev (tym83)
as code owners
September 7, 2026 13:24
Andrei Kvapil (kvaps)
added a commit
to cozystack/cozystack
that referenced
this pull request
Sep 7, 2026
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Andrei Kvapil (kvaps)
added a commit
to cozystack/cozystack
that referenced
this pull request
Sep 7, 2026
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Andrei Kvapil (kvaps)
added a commit
to cozystack/cozystack
that referenced
this pull request
Sep 7, 2026
…3931) ## What this PR does Adds tenant self-service import of virtual machines from VMware vSphere, as a pair of CRDs in a new `migration.cozystack.io` group reconciled by a controller in core. Konveyor Forklift is the transfer engine underneath; tenants never see it. A `VMImportSource` registers a long-lived connection to a provider — type, endpoint, credentials entered on the spec and materialized into a Secret by the controller, because tenants cannot create Secrets in Cozystack. A `VMImportTask` runs the one-shot operation: reference a source, name the machines, get `VMDisk`s and `VMInstance`s. The outputs carry no owner reference back to the task, so deleting a finished import removes the migration machinery and nothing else. This supersedes the chart-based tenant API in #1982 while keeping its Forklift packages unchanged, carried over here with attribution. The design is written up in cozystack/community#62, which records why a Helm chart lost to a controller: `apps.cozystack.io` is a projection over HelmReleases, so "the outputs survive deletion" would need `helm.sh/resource-policy: keep`; a completed migration is terminal while Flux reconciles toward a steady state forever; and the number and size of the disks an import produces is only known after the source VM has been inspected, which a template cannot express. Two things are worth calling out for review. **The controller ships no vSphere client.** Forklift already authenticates, validates VM references and reports unmapped networks and datastores, and the KubeVirt VirtualMachine it builds carries the source VM's CPU, memory and disk order — so the inventory is read from Forklift's own objects rather than from a second client that could disagree with the one doing the work. A tenant sees Forklift's verdicts verbatim rather than a paraphrase. **The volume handoff copies nothing.** The transferred PersistentVolume is retained and re-bound into the claim a `VMDisk` expects, with `cdi.kubevirt.io/storage.populatedFor` telling CDI the claim is already populated — the same primitive the Velero VM-restore path already depends on in production. That closes both the raw-PVC gap raised in review on #1982 and the duplicate-copy waste tracked in #3924. Everything is opt-in: three packages added to `bundles.enabledPackages`, and one platform value (`vmImport.vddkImage`) naming the VDDK image, which Cozystack cannot ship because it is built from a proprietary VMware SDK. Empty is a supported state and makes a vSphere source report itself unavailable at registration rather than mid-transfer. ### Validation Run on a live Cozystack v1.6.2 cluster and rolled back afterwards. Confirmed working end to end: CRD install and printer columns, the VDDK policy (including that no credentials Secret is projected for a source that cannot work), credential projection with the ownership guard, the Provider pair with the operator's image stamped on it, the local-cluster destination Provider reaching Ready/Connected/Inventory, Forklift's connection verdict mirrored onto the source verbatim, the diagnostic chain from Forklift through the source to the task, that nothing is created while a source is unready, and that deleting a source garbage-collects both Providers and the Secret. That testing found four defects, none of which were visible from reading code, and all are fixed here: the task controller could not start at all when Forklift is absent (a watch on an absent CRD never syncs, so controller-runtime refused to start it and the pod sat at 0/1); a vSphere secret needs a CA certificate rather than a thumbprint, so as first modelled every source would have sat at `SecretNotValid` forever with a reason that reads like a credentials problem; Forklift's absence was returned as a reconcile error, so a source kept a stale condition indefinitely while the controller hot-looped; and the controller's own Deployment violated the restricted Pod Security Standard, which is indefensible for a feature whose design argument is that an import stays inside it. A further four assumptions were corrected against operational experience from the original implementation's work on a real vSphere environment, which this session had no access to. Each failed silently: the VirtualMachine Forklift creates is labelled with the Plan's **UID**, not its name, so the disk would never have been adopted; the transfer arrives as a CDI-populated claim with no DataVolume, carrying an owner reference to that VirtualMachine, and clearing it was conditional on a DataVolume existing — on the original implementation garbage collection removed such a claim 357ms after the VM went, destroying a successfully migrated disk; the terminal signal is the phase the engine settles on, not only a completion timestamp; and progress read from a name-matched pipeline step pegs at zero if a release renames it. **What could not be tested:** everything past authenticating to vCenter — inventory resolution, the loop that learns network and datastore mappings from Forklift's validation conditions, the transfer itself, and this controller's own code path through the handoff. No vCenter was available. The handoff mechanics were separately proven by hand on the same cluster in earlier work; what remains unproven is this implementation of them. The mapping loop is the least-supported part of the change and is the thing to look at hardest. ### Screenshots https://github.com/user-attachments/assets/f313f6ba-8e72-42d9-b65a-033866ad39d5 ### Downstream repositories Walked the trigger map against the diff. `cozystack/website` is reached on two counts and the follow-up is open as [cozystack/website#673](cozystack/website#673), a migration guide covering the pre-flight checks, the administrator's VDDK build, and the driver caveat the lab run surfaced: - `packages/core/platform/values.yaml` gains `vmImport.vddkImage`, and `content/en/docs/next/operations/configuration/platform-package.md` is a hand-written table of those keys. - Three platform components are added, which reaches `guides/platform-stack/_index.md` and `operations/configuration/licenses.md` — the latter is where the proprietary VDDK image belongs, since Cozystack references it without shipping it. The others were checked and are not affected: no package is added under `packages/apps/` or `packages/extra/` (terraform-provider, website app lists), no key the Ansible role sets changes, nothing under `hack/` is moved or renamed and no make target changes behaviour (ccp, external-apps-example), and no node prerequisite changes (talm). - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#673 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(migration): import virtual machines from VMware vSphere into a tenant. A VMImportSource registers a connection to vCenter and a VMImportTask imports named machines, producing ordinary VMDisks and VMInstances that survive deletion of the import. Opt-in: add cozystack.forklift-operator, cozystack.forklift and cozystack.migration-controller to bundles.enabledPackages, and set vmImport.vddkImage to a VMware VDDK image you build yourself — Cozystack cannot redistribute it. Guest conversion and warm migration are not included in this version. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added vSphere VM migration support, including source connections, import tasks, inventory discovery, progress tracking, and conversion into Cozystack VMs and disks. * Added dashboard pages for managing migration sources and imports, with source-dependent VM selection and progress details. * Added optional VM firmware settings for BIOS/UEFI, Secure Boot, and persistent EFI storage. * Added opt-in platform packages for migration services and Forklift integration. * **Security** * Restricted access to the ingress admission webhook port with a network policy. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a migration guide for the
forklift.cozystack.ioimport API: register a vCenter connection, name the machines, getVMDiskandVMInstanceobjects.Placed in
next, not a released version: the API it documents is still in review (cozystack/cozystack#3931, design in cozystack/community#62). It should not be published until that lands.The guide is written from an end-to-end migration run against a real vSphere environment rather than from the API surface, which is why the pre-flight section is as long as it is. Each check in it corresponds to a failure that actually happened, and every one of them surfaces late and blames something else:
WaitForFirstConsumerstorage class deadlocks the transfer — and it is frequently the cluster default;There is also an administrator section on building the VDDK image. The kit is licensed software that VMware distributes to registered users only and nobody may redistribute, which is why Cozystack ships no image and the platform value starts empty — leaving an administrator with a build step and, until now, no instructions for it. It covers matching the kit version to vSphere, building for
linux/amd64(an arm64 build pulls and then fails to execute on the workers), making a private registry pullable from the namespaces that need it, and confirming the pull before the first migration rather than during it.The rest documents what the result actually is: the transfer is cold, so the source is powered off; nothing is copied twice; the imported VM starts
Halted; and the outputs outlive the task that produced them. Limitations are stated explicitly — vSphere only, cold only, one storage class per task, pod networking, no guest conversion — so nobody plans a Windows migration around a path that copies disks as-is.