Repository navigation
Daily Operations
The tasks you perform once MCOS is running. Health checks, adding and removing pools, looking at activity, applying configuration changes.
For first-time install, see Quick Start. For maintenance like updates and uninstall, see Maintenance.
Run after a host reboot, after a config change, or any time something feels off.
Open http://localhost:7300/ → Overview. Five summary cards should render:
- MCP Gateway: state and health
- Worker Pools: count + ready instances + active leases
- Connected Clients: count
- Governance Posture: pass / warn / blocked
- Host Telemetry: CPU / memory / disk
If any card is empty or shows an error, drill into the matching destination.
# Service status
Get-Service MasterControlProgram | Format-Table -AutoSize
# Health endpoint (operator surface)
Invoke-RestMethod http://localhost:7300/api/health | ConvertTo-Json
# Discovery doc (what's advertised on the LAN)
Invoke-RestMethod http://localhost:7300/api/discovery | ConvertTo-Json -Depth 6
# Gateway state + health probe (live IMcpGateway::Probe)
Invoke-RestMethod http://localhost:7300/api/gateway/status | ConvertTo-Json
Invoke-RestMethod http://localhost:7300/api/gateway/health | ConvertTo-Json
# Bootstrapper preflight (validates payload + service + firewall + registry)
& "C:\Program Files\Master Control Orchestration Server\MasterControlBootstrapper.exe" preflight --json-outputA green health check means: service Running, /api/health returns status=ok, gateway state is running or configured, preflight reports zero issues.
The presence roster is heartbeat-driven. Clients that POST to /api/telemetry/heartbeat show up here.
Dashboard → Clients. Each row is one client with:
-
clientId,clientType, IP - Self-reported CPU%, memory%, GPU%, GPU MB
- Bytes sent/received per second
- Last seen timestamp
Cells reading unavailable mean the client never reported that metric. Honest by design — never collapsed to 0%.
Invoke-RestMethod http://localhost:7300/api/telemetry/clients | Format-Table clientId, clientType, ipAddress, lastSeenUtc -AutoSizeIf a client you expect is missing, see Troubleshooting §LAN discovery — the host probably can't reach MCOS.
Two parallel rings:
PHASE-08 events. 1024-entry capped ring. Categories: system / gateway / worker / client / discovery / governance × severities: info / warning / error / critical.
Invoke-RestMethod 'http://localhost:7300/api/telemetry/events?max=50' | ConvertTo-Json -Depth 6Dashboard surface: Activity destination → top card.
ADR-001 ring. Operator-surface events (client registration, governance decisions, exports).
Invoke-RestMethod http://localhost:7300/api/activity | ConvertTo-Json -Depth 6Dashboard surface: Activity destination → bottom card.
$body = @{
poolId = 'mcos-shell-tools'
kind = 'mcp-server'
logicalMcpUrl = 'http://localhost:18443/mcp/shell'
template = @{
executable = 'C:\Program Files\my-mcp-shell\my-mcp-shell.exe'
args = @('--port', '18443')
workingDirectory = ''
environment = @{}
transport = 'streamable_http'
healthProbe = @{
transport = 'http'
path = '/health'
intervalMs = 5000
timeoutMs = 1500
unhealthyThreshold = 3
}
}
scalePolicy = @{
minInstances = 0
maxInstances = 4
maxActiveLeasesPerInstance = 8
scaleOutQueueWaitMs = 1500
scaleInIdleSeconds = 120
}
drainPolicy = @{
drainStickySessions = $true
drainTimeoutSeconds = 30
routeNewSessionsToReplacement = $true
}
} | ConvertTo-Json -Depth 6
Invoke-RestMethod -Method POST `
-Uri http://localhost:7300/api/pools `
-ContentType 'application/json' `
-Body $bodyPOST /api/pools is upsert — running it again with the same poolId updates the definition.
After registering: dashboard → Pools → confirm the new pool card appears with instanceCount: 0 and at saturation: no.
Invoke-RestMethod -Method POST http://localhost:7300/api/pools/mcos-shell-tools/scaleOr click Scale to min on the pool card.
The supervisor spawns instances under Job Objects. Each instance transitions Configured → Starting → Ready (or Failed). Dashboard reflects state in the per-pool card.
For full pool semantics see Worker Pools.
Invoke-RestMethod -Method POST http://localhost:7300/api/pools/mcos-shell-tools/drainOr click Drain on the pool card.
What happens:
- Every instance transitions to
Draining. - Existing sticky leases continue routing to their bound instance.
- New stateless leases route to non-draining Ready instances elsewhere.
- As leases drain to zero, instances transition to
Stopped.
Hot-migration is forbidden — stateful sessions complete on their original instance. ADR-002 §8 / FORBIDDEN-CONTRACT §2.4.
Invoke-RestMethod -Method POST http://localhost:7300/api/pools/mcos-shell-tools/removeRemoving a pool also removes the persisted definition from the configuration file. Existing in-flight leases on running instances complete first; the supervisor reaps the workers under Job Object closure.
# All pools, all leases
Invoke-RestMethod http://localhost:7300/api/pools | ConvertFrom-Json | ForEach-Object {
$poolId = $_.poolId
$leases = Invoke-RestMethod "http://localhost:7300/api/pools/$poolId/leases"
[pscustomobject]@{ Pool = $poolId; ActiveLeases = $leases.Count }
} | Format-Table -AutoSize
# Saturation snapshot for one pool
Invoke-RestMethod http://localhost:7300/api/pools/mcos-shell-tools/saturation | ConvertTo-JsonDashboard surface: Pools destination → expand the N active leases disclosure on each card.
When a governed action would be high-impact, CLU defers it instead of executing immediately. The pending list shows up under Governance.
Dashboard → Governance → "Pending approvals" card. Each row has Approve / Reject.
# List pending
Invoke-RestMethod http://localhost:7300/api/clu/approvals |
Where-Object { $_.status -eq 'pending' } |
Format-Table id, action, actor, createdAtUtc -AutoSize
# Approve
Invoke-RestMethod -Method POST http://localhost:7300/api/clu/approvals/<id>/approve
# Reject (with reason)
$body = @{ reason = 'Out of policy this quarter' } | ConvertTo-Json
Invoke-RestMethod -Method POST -Uri http://localhost:7300/api/clu/approvals/<id>/reject `
-Body $body -ContentType 'application/json'See Governance for the full operator approval queue + decision policy.
# Three platforms supported
foreach ($p in 'windows','macos','ios') {
$bundle = Invoke-RestMethod "http://localhost:7300/api/governance/bundles/$p"
$bundle | ConvertTo-Json -Depth 6 |
Out-File "$env:USERPROFILE\Desktop\mcos-governance-$p.json" -Encoding utf8
Write-Host "Saved $p bundle (sha256=$($bundle.checksum))"
}Dashboard surface: Governance → "Governance bundles" card has tabs and a download link per platform. See CLU Governance.
If the field is on the WinUI Settings panel, edit there and click Apply Host Settings. Otherwise:
# Edit the current configuration file.
notepad "$env:ProgramData\MasterControlOrchestrationServer\config\master-control-orchestration-server.json"
# Restart so the runtime re-reads
Restart-Service MasterControlProgramSome changes hot-apply through POST /api/config or a partial PATCH /api/config request. Unsafe network posture changes require X-Confirm-Unsafe: true. See Configuration for the field-by-field reference.
Diagnostics logs land under %PUBLIC%\Documents\Master Control Orchestration Server\logs\runtime\.
$log = "$env:PUBLIC\Documents\Master Control Orchestration Server\logs\runtime\events.jsonl"
Get-Content $log -Wait -Tail 50
# Or filter by category
Get-Content $log -Wait -Tail 0 | Where-Object { $_ -match '"category":"gateway"' }Dashboard equivalent: Activity destination polls every 5 seconds.
- First-time install → Quick Start
- Field-by-field config → Configuration
- Pool model in detail → Worker Pools
- Telemetry surface → Telemetry and Activity
- Governance roles → CLU Governance (bundles) + Governance (approvals)
- Updates and uninstall → Maintenance
- What to do when something breaks → Troubleshooting
Source for this wiki is maintained under docs/wiki/ in the Master-Control-Orchestration-Server repository.
Setup
Connect Clients
Operate
- Daily Operations
- Dashboard
- Telemetry and Activity
- Gateway
- Worker Pools
- Sub-Agents
- Governance
- CLU Governance
- Privileges
Maintain
Troubleshoot
Reference