Developer portal + governed compute-mesh plane (closes #17) - #18
Merged
Conversation
…loses #17) Two views over one governed surface, plus the plane that makes a low-mem box a front-end to the whole mesh. Developer portal (move #8) — tools/portal_server.py: - dependency-free, scale-to-zero, read-only web console (stdlib http.server, no external CDN; inline HTML/CSS/JS, dark theme) - pure route() core (unit-tested): / console, /healthz, and /api/{capabilities, lifecycle,evidence,compute} over the SAME data agents see via the MCP surface - one governed source, two views: agent via MCP, human via portal; mutations only ever flow through the MCP surface + the fail-closed promotion gate, never here Compute-mesh plane — tools/compute_plane.py: - one governed door to any substrate: local, k8s, HPC/SLURM, WASM edge, p2p mesh, volunteer compute (BOINC/Folding@home/open-HEP-style), RLC-style blockchain market - place(workload, policy, availability): routes by per-project/per-account policy + live mesh availability; scales out where it can (highest available elasticity) - GOVERNED / fail-closed: sensitive work never lands on an untrusted backend; blocks rather than silently degrading (won't run GPU work on a non-GPU local box); every placement sealed into a tamper-evident receipt - surfaced in the portal dashboard (compute-mesh section, trust-annotated) CapD + wiring: - capd/compute-plane.mesh.capd.json (caps.compute.mesh-plane@0.1.0) - capd/devspace.local-dev.capd.json now composes_with the compute plane - validate.py: portal + compute plane + both CapDs added to REQUIRED, with per-CapD key/id-drift checks - Makefile: `portal`, `compute`, `tools-test` targets Tests: 12 compute-plane + 7 portal (26 tools tests total, all green); portal smoke- tested end-to-end over HTTP.
… the mesh
Extends the compute-mesh plane with the live-availability layer and the cloud-shell
fog spec's zero-trust attach flow, and puts the whole app suite on the one mesh.
Live mesh telemetry — tools/mesh_telemetry.py:
- fail-closed liveness registry: nodes emit heartbeats (backend + capacity); the
registry sums LIVE capacity per backend and expires stale nodes by TTL. No
heartbeat -> zero capacity -> the plane won't schedule there.
- file-backed (sovereign, no broker); the read-only portal reads, never ingests.
- this is the spec's HyperSwarm discovery / "find candidate nodes".
Zero-trust MCP-A2A grants — tools/mcp_a2a_grant.py:
- Policy Authority issue_grant(): Attest (TPM/TEE + cosign required) -> Decide (a real
scheduled placement) -> Grant (signed, session-bound, constraint-carrying; quorum
proof when required). No attestation / blocked placement / missing quorum -> refused.
- fog-node Policy Gate verify_grant(): re-verify signature + session binding + expiry
+ attestation + that the specific PTY/FS op is in-constraints, on attach and on every
op. Fail-closed. HMAC stands in for the Key Authority (HSM/KMS) — swap the interface.
App suite on the mesh — mesh/suite-workloads.json:
- Noetica, memory-mesh, TurtleTern, Goose Notes, BearBrowser as first-class workloads,
each with the policy that reflects its real governance shape (offensive tooling never
rides the volunteer grid; sensitive reasoning stays trusted+attested).
- portal /api/placements runs place() for each against LIVE availability.
Cloud-shell fog spec, first-class:
- capd/cloudshell-fog.capd.json (caps.compute.cloudshell-fog@0.1.0) + docs/CLOUDSHELL_FOG.md
witness every spec box -> real code, and the 0..11 attach flow.
Portal: /api/mesh (per-node liveness) + /api/compute now live + /api/placements; console
gains a live telemetry summary + "app suite on the mesh" section.
Wiring: validate REQUIRED + CapD checks; Makefile mesh-demo/grant targets; heartbeats dir
gitignored (runtime). Tests: +8 telemetry +12 grant, 48 tools tests green.
…chemas
The first cut invented its own grant shape (flat session_id/subject, allowed_ops
constraints, epoch floats, a bare HMAC signature). That diverged from the estate's
zero-trust authority. Reshaped to emit/consume the CANONICAL contracts owned by
SourceOS-Linux/mcp-a2a-zero-trust:
- Grant: grant_id + ISO issued_at/expires_at + binding{spiffe_id,aum_digest,
session_id} + capability{kind,capability_ref,capability_digest,effect,executor_ref}
+ constraints + policy_hash + optional quorum_proof + evidence_refs + sig{issuer,sig}.
Matches examples/grant.example.json exactly (additionalProperties:false).
- AttestationBundle {subject,results{tpm_valid,cosign_valid,fido2_valid},evidence_refs}
consumed as the Attest input.
- QuorumProof {rule,validators,signed_payload_hash,signatures[{kind,spiffe_id,sig}]}.
- verify_grant returns a canonical tool_grant.validate result {valid,expired,revoked,
reason} (examples/tool_grant_check.example.json).
Conformance is enforced, not asserted: schemas/a2a/ vendors the canonical schemas
hash-pinned to the authority's schemas/index.json (verified on vendor; see
schemas/a2a/PROVENANCE.md), and test_mcp_a2a_grant.py validates every emitted Grant /
QuorumProof / AttestationBundle against them with a dependency-free schema checker.
docs/CLOUDSHELL_FOG.md + capd/cloudshell-fog.capd.json now name mcp-a2a-zero-trust as
the shape authority. 50 tools tests green (14 grant incl. 3 conformance).
…Ops + semantic-action, one plane
Zenodo-style citable, content-addressed, reproducibility-graded layer over the estate.
Folds three demonstrated systems into one governed plane:
- Zenodo/reproducible-fusion: mint_id() content-addresses every record as a citable
commons:<domain>/<name>@<version>+<digest>; deterministic (same inputs -> same id).
- ARM (Asset Reuse Manager): domain/category/asset_type navigation, recommend() by
reuse score, record_use() use/evaluate feedback loop.
- MLOps reproducibility: fail-closed reproducibility GATE — a record may claim
`reproducible` only if provenance carries source_digest AND (attestation_ref OR
sbom_digest); else honestly `declared`. The dashboard never overstates.
- Semantic API: records carry an optional declarative semantic_action (signature +
ontology/policy constraints).
The estate ingests itself: estate_commons() deposits every capd/*.capd.json + the suite
workloads, so caps.compute.mesh-plane, caps.compute.cloudshell-fog, the devspace inner
loop, and the five app-suite workloads are all first-class citable records — the commons
even contains itself.
tools/commons.py + test_commons.py (8 tests). Portal /api/commons + console section
(honest reproducible/declared split). capd/knowledge-commons.mesh.capd.json +
docs/KNOWLEDGE_COMMONS.md (witnesses ARM/MLOps/Semantic-API -> implementation). validate
REQUIRED + CapD checks; Makefile commons target. 58 tools tests green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ships the developer portal (superiority-march move #8) and the governed compute-mesh plane — the piece that lets a low-mem box (this M2) act as a front-end to any compute the mesh offers, seamlessly and under policy.
Developer portal —
tools/portal_server.pyhttp.server, no external CDN (inline HTML/CSS/JS, dark theme).route()core (unit-tested):/console,/healthz, and/api/{capabilities,lifecycle,evidence,compute}— the same data agents see via the MCP ops surface.Compute-mesh plane —
tools/compute_plane.pylocal,k8s,hpc-slurm,wasm-edge,p2p-mesh,volunteer-boinc(BOINC/Folding@home/open-HEP-style),blockchain-rlc.place(workload, policy, availability)routes by per-project / per-account policy + live mesh availability, scaling out where it can (highest available elasticity).CapD + wiring
capd/compute-plane.mesh.capd.json(caps.compute.mesh-plane@0.1.0);devspace.local-devnowcomposes_withit.validate.py: portal + compute plane + both CapDs added toREQUIRED, with per-CapD key/id-drift checks.Makefile:portal,compute,tools-testtargets.Tests
/healthz,/api/compute,/api/capabilities,/all 200).How it maps to the demonstrated designs
BOINC computing-preferences (per-project/per-account volunteer config), IBM Parallel Environment (HPC toolkit / SLURM), Docker volume drivers (pluggable substrate), Istio mesh — met and bettered: one governed plane across all substrates instead of per-substrate config silos.
Closes #17.