Hyper V Discovery Developer Guide

Hyper-V discovery - Developer Guide

How the virtual estate (#199 Parts A and B) is built, and the rules a change must keep. The user page is Hyper-V and Failover Cluster discovery; the reasoning is in the design.

In one breath: a read-only PowerShell collector pushes a report per scope. One service (VirtualEstateService) applies it to ten virt_* tables, under the inventory ingest's rules: observed time, complete/partial sections, stale/replay/conflict, a savepoint per section, removal is not deletion. Identity changes are raised for a person. Each connection is a pushed source on job health. The tables are named for the estate, not Hyper-V, so vCenter, Proxmox and Cloud Director can move onto them, and Parts B and C are built once.

---

1. The files

🗄️ schema · 🧠 rules · 🔌 REST · 🖥️ UI · 🩺 job health · 🪟 collector · 🧪 tests

🎨	File	What it does there
🗄️	database/freeitsm.sql, includes/db_verify_schema.php, includes/db_verify_indexes.php, api/system/db_verify.php, includes/db_verify_modules.php	Ten virt_* tables; children cascade from virt_connections and virt_vms. virt_ maps to Assets
🧠	includes/services/virtual_estate.php	submit(), the six apply_*() sections, decideReview(), mintKey(), freshness()
🔌	api/v1/resources/virtual_estate.php, api/v1/lib/routes.php, api/v1/lib/permissions.php, api/v1/spec.json	POST /virtual-estate/{id}/reports (only the connection's api_key_id), GET /virtual-estate/{id}; permission virtual_estate
🖥️	api/assets/hyperv_connections.php, asset-management/settings/hyperv.js, asset-management/settings/index.php, asset-management/settings/manifest.php	Assets → Settings → Hyper-V (Cap::ASSETS_HYPERV, sensitive: it mints keys)
🖥️	api/assets/get_servers.php, asset-management/servers/index.php	Hyper-V rows (scoped to the analyst's companies), the reports panel, hosts and clusters
🩺	includes/job_registry.php (jobRegistryAll()), includes/job_health.php, includes/job_runs.php, api/reporting/job_health.php	One virt_push_<id> entry per active connection; trigger push
🪟	scripts/Invoke-HyperVInventory.ps1	The collector
🧪	tests/virtual-estate.php	47 checks - every scenario in criterion 7, REST key gating, job health
🧪	tests/hyperv-collector.php, tests/hyperv-collector/Mock-HyperV.ps1	The REAL collector against mocked cmdlets, its reports through the REAL ingest, and sending to the local API (Windows only)
🧪	tests/virtual-estate-load.php, tests/fixtures/virtual-estate/synthetic.php	Sizing on the synthetic reference estates; the fixture generator
📄	samples/hyperv/	Fully synthetic example reports and their generator

---

2. The tables

Table	Identity (external_id)	Notes
virt_connections	-	One company (tenant_id); api_key_id is the only key accepted; interval, grace, retention
virt_clusters	Get-Cluster Id	quorum type, witness
virt_hosts	hardware UUID, or name:<host>	identity_kind source / name; cluster_id; node_state
virt_cluster_roles	Get-ClusterGroup Id	owner host, preferred owners
virt_vms	Get-VM Id	host, cluster, is_clustered (NULL = unknown), role, possible_owners_json (host row ids)
virt_vm_disks, virt_vm_networks	per VM	replaced as a set by a complete section; observed_at guards against older reports
virt_placements	-	one open row per VM; a host change closes it and opens another
virt_reports	-	every report received and what each section came to
virt_identity_reviews	-	open → same / different / superseded

Every discovered row carries first_seen_at, last_seen_observed_at, not_seen_since and not_confirmed_since.

---

3. A report, step by step

submit(connection, report)
  envelope(): schema_version 1, observed_at with a timezone (not in the future), scope, sections
  scope_key = hash of the scope; fingerprint = hash of scope + sections + unreadable
  newest applied report for this scope:  newer -> stale  |  same time, same hash, applied -> replayed
                                          same time, different hash -> 409 observation_conflict
                                          same time, same hash, partial -> apply again (retries the failed sections)
  JobRun::start('virt_push_<id>', 'push')
  one transaction; per section a SAVEPOINT:  clusters, hosts, cluster_roles, vms, vm_disks, vm_networks
  markUnconfirmed()  raiseIdentityReviews()  purge()  commit
  record the report; connection's last_report_at, last_outcome, last_summary
  200 applied | 207 partial | 422 failed

The rules each section keeps

• Upsert by (connection_id, external_id), reading the connection's rows once per section (FOR UPDATE). An object last observed after this report is skipped (stale_objects), so an older report sent under a different scope can never wind it back.
• Complete marks not-seen, within scope only. markNotSeen() limits itself to what the report's scope covers - its clusters, and the hosts in them or named as standalone - and skips anything on an unreadable host or cluster (onUnreadable()).
• Unreadable is "not confirmed", never "gone" (markUnconfirmed()).
• Hosts are referenced by NAME inside a report (a VM's host, a role's owner, possible owners), resolved per report against the hosts just applied. That is what lets a host rename keep its VMs.
• Hardware UUIDs are checked: all-zero, all-F, malformed, or the same UUID on two hosts in one report → name: identity.
• A bad item fails its section, rolled back to the savepoint, not skipped. Unknown VMs in vm_disks / vm_networks fail the section rather than dropping data.

Identity reviews

raiseIdentityReviews() pairs a row that is not seen with a live row of the same kind, same name (case-insensitive), first seen at or after the old one was last seen. For hosts, it also pairs two name-identity hosts in the same cluster. Raised once per pair.

decideReview(same) keeps the old row - its id, first seen and placements - and gives it the new identity and the new row's current facts. It moves the new row's disks, adapters, placements and anything pointing at it, then deletes the new row. It refuses (review_changed, case closed as superseded) if the old row is seen again or the new one goes before the decision.

---

4. Job health for a pushed source

jobRegistryAll($conn) = the scheduled jobs + one entry per active connection: key virt_push_<id>, 'pushed' => true, interval and grace from the connection, its own name (name_params). Runs are recorded by submit() with trigger push, and the freshness rule counts push beside cron and web. An unreadable host makes the run partial.

If the connections table cannot be read, health shows a Pushed sources: unknown entry, never silence.

---

5. The collector

Windows PowerShell 5.1 syntax throughout (no ??, no ternary). It only calls Get-* and CIM reads. Per cluster it reads Get-Cluster, Get-ClusterQuorum, Get-ClusterNode and Get-ClusterGroup. For each "Virtual Machine" resource it reads VmID (Get-ClusterParameter) and its possible owners (Get-ClusterOwnerNode). Per host it reads the UUID (Win32_ComputerSystemProduct), the OS, Get-VMHost, Get-VM, disks (Get-VMHardDiskDrive + Get-VHD) and adapters (+ VLAN). A host or cluster that throws goes into unreadable. A VM detail that throws is null.

-TestAccess records every read through Try-Read and prints the table, exiting 2 if anything was refused. -Redact uses an HMAC keyed on %ProgramData%\FreeITSM\redact.key: names become host-1a2b3c4d, ids become stable fake GUIDs, error text is replaced outright.

Reports go to <ApiUrl>/api/v1/index.php?path=/virtual-estate/<id>/reports with Authorization: Bearer, which works without URL rewriting.

---

5b. Part B - source-bound CIs

includes/services/virtual_estate_cmdb.php (VirtualEstateCmdbService), with api/assets/hyperv_cmdb.php (the panel) and api/cmdb/discovery_source.php (the CI's Discovered card).

Table	What it is
virt_cmdb_mappings	per connection: classes per kind, the four relationship types, sync_names, enabled
cmdb_object_sources	bindings as periods - one live per CI and per object, start_source / end_reason (unbound, ci_deleted, rebound). Same shape as asset_ci_links
cmdb_relationship_sources	the relationships the sync created - the only ones it ever changes or removes
virt_cmdb_decisions	create_new, leave_unbound, ci_deleted - what the sync must not decide alone

status() decides each object: bound → keep (rename if allowed); open identity review as the new object → waiting; a decision → as decided; a same-name unbound CI of the mapped class in the company → match (never bound by name); otherwise create.

sync(): one transaction, a savepoint per CI (a class requiring properties fails that object only), then syncEdges(). It wants a set of edges from the virt rows and diffs it against the edges it owns. It does not adopt an identical edge a person drew, and it keeps owned edges of objects that are not seen. Workflow events fire after the commit. VirtualEstateService::submit() calls it after the report's own commit, so a CMDB problem never loses a report.

Hooks: CmdbService::deleteObject() → ciDeleted(); the connection delete in api/assets/hyperv_connections.php → connectionDeleted() (live bindings end as connection_deleted, rows kept - forCi() LEFT JOINs the connection; mapping, decisions and cmdb_relationship_sources deleted, as none has a foreign key); VirtualEstateService::rebind() → rebound() (the old row keeps its binding; a binding only the new row had moves; if both had one, the new one ends as rebound); purge() skips bound objects via boundIdsSql(); cmdbMemberState() → memberState() and cmdbPossibleOwners() → possibleOwnerCis() feed Part C.

🧪 tests/virtual-estate-cmdb.php - 32 checks, from the preview to unbinding.

6. Traps

• TRAP: R is an alias. The redaction helper was first called R. r is PowerShell's alias for Invoke-History, and an alias beats a function, so every R 'host' $name ran Invoke-History. It is Mask now. Check Get-Alias <name> before naming a short function.
• TRAP: PowerShell scoping is dynamic. A mocked cmdlet that read $uuid saw the collector's $uuid variable, not the mock's own table. The mock's tables are named $Mock* for that reason.
• TRAP: comma binds tighter than +. @('10.0.0.' + $n, 'fe80::1') is ONE string. Parenthesise.
• TRAP: jobRegistry() is the scheduled jobs only. Anything listing or looking up jobs must use jobRegistryAll($conn).
• TRAP: Part B binds CIs to virt row ids. decideReview() keeps the old row for that reason, and purge() skips a bound object.
• TRAP: a person who deletes a relationship the sync owns sees it come back on the next report - discovery is the authority for its own edges. Unbinding the object is how to stop that.
• TRAP: api/system/db_verify.php $primaryKeys. A new table whose key is not id must be listed there, or Database Verification fails to create it. virt_cmdb_mappings, cmdb_relationship_sources and virt_cmdb_decisions are.
• TRAP: the merge guard reads every host_id column as an asset reference, because Software's is one. virt_vms.host_id and virt_placements.host_id are in AssetMergeService::NOT_REFERENCES.
• Size. On this dev box: 800 VMs ≈ 1 s, 8,000 VMs ≈ 9 s in one 7.5 MB report (tests/virtual-estate-load.php). Recommend -ReportPer Cluster.

---

7. What is not proven

The collector's reads have only met mocked cmdlets. The property names it relies on - IsClustered, MemoryAssigned, OwnerGroup.Id, the VmID parameter, Win32_ComputerSystemProduct.UUID - are Microsoft's documented ones. Whether they hold on every Windows Server version, and whether the narrower permissions are enough, can only be confirmed on a real estate. The user page says so.