From 04946f4da85289f5c1f0f1627f0ca64948962fcd Mon Sep 17 00:00:00 2001 From: coodos Date: Tue, 7 Jul 2026 20:11:37 +0530 Subject: [PATCH 1/4] feat: synthesise skill --- skills/README.md | 35 +++ skills/w3ds/SKILL.md | 91 +++++++ skills/w3ds/reference/dev-setup.md | 227 +++++++++++++++++ skills/w3ds/reference/evault.md | 342 +++++++++++++++++++++++++ skills/w3ds/reference/identity.md | 125 ++++++++++ skills/w3ds/reference/platform.md | 387 +++++++++++++++++++++++++++++ skills/w3ds/reference/protocols.md | 330 ++++++++++++++++++++++++ skills/w3ds/reference/registry.md | 160 ++++++++++++ skills/w3ds/reference/wallet.md | 285 +++++++++++++++++++++ 9 files changed, 1982 insertions(+) create mode 100644 skills/README.md create mode 100644 skills/w3ds/SKILL.md create mode 100644 skills/w3ds/reference/dev-setup.md create mode 100644 skills/w3ds/reference/evault.md create mode 100644 skills/w3ds/reference/identity.md create mode 100644 skills/w3ds/reference/platform.md create mode 100644 skills/w3ds/reference/protocols.md create mode 100644 skills/w3ds/reference/registry.md create mode 100644 skills/w3ds/reference/wallet.md diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 000000000..33356f0f6 --- /dev/null +++ b/skills/README.md @@ -0,0 +1,35 @@ +# MetaState Skills + +Installable Claude Code skills for the MetaState / W3DS ecosystem. + +## Available skills + +| Skill | Purpose | +|-------|---------| +| [w3ds](./w3ds) | Web 3 Data Spaces — build post-platforms, call the eVault GraphQL API, wire the Web3 Adapter, implement `w3ds://auth` / `w3ds://sign`, debug local dev. | + +## Install + +Skills in this directory follow the [skills.sh](https://skills.sh/) `/@` layout and are installable via the `skills` CLI: + +```bash +npx skills add MetaState-Prototype-Project/prototype@w3ds -g -y +``` + +`-g` installs into `~/.claude/skills/`. Omit for a project-local install. + +## Local development + +To hack on a skill without publishing, symlink it into the Claude skills directory: + +```bash +ln -s "$(pwd)/skills/w3ds" ~/.claude/skills/w3ds +``` + +Restart Claude Code so the new skill is picked up. Edits to files under `skills/w3ds/` take effect immediately on the next skill invocation — no re-symlink needed. + +## Authoring notes + +Each skill is a directory with a top-level `SKILL.md` and optional `reference/` files. The `SKILL.md` frontmatter needs at minimum a `name` and a `description`; the description is what Claude uses to decide when to trigger the skill, so it should list the concrete surfaces the skill covers (concepts, APIs, protocol names, common questions). + +Keep the main `SKILL.md` scannable (~150 lines) and push deep content into `reference/*.md` files that get loaded on demand. diff --git a/skills/w3ds/SKILL.md b/skills/w3ds/SKILL.md new file mode 100644 index 000000000..16208c040 --- /dev/null +++ b/skills/w3ds/SKILL.md @@ -0,0 +1,91 @@ +--- +name: w3ds +description: "Use when the user is building on Web 3 Data Spaces (W3DS) or the MetaState prototype — building a post-platform, integrating an eVault, calling the eVault GraphQL API (createMetaEnvelope, updateMetaEnvelope, removeMetaEnvelope, bulkCreateMetaEnvelopes, uploadFile, bindingDocument*), wiring the Web3 Adapter, writing a webhook controller for /api/webhook, authoring mapping.json files, using the wallet-sdk, implementing the w3ds://auth or w3ds://sign flow, resolving W3IDs / eNames via the Registry, working with the Ontology service, dealing with Binding Documents, dereferencing w3ds://file URIs, provisioning an eVault, syncing public keys, or debugging local dev (Registry, Provisioner, eVault-core, Dev Sandbox, pnpm dev:core). Also use when the user asks what an eVault, W3ID, eName, MetaEnvelope, Envelope, Ontology, Web3 Adapter, Awareness Protocol, or Awareness-as-a-Service is." +license: Apache 2.0 +--- + +# W3DS — Web 3 Data Spaces + +W3DS lets users own their data in a personal **eVault** while platforms act as interchangeable frontends. Data written on one platform automatically syncs to every other registered platform via the **Awareness Protocol**. This skill is for developers building on W3DS: integrating platforms, calling the eVault GraphQL API, wiring the Web3 Adapter, and debugging local dev. + +## Ecosystem map + +The "digital self" is a triad: **eName + eID certificate + eVault**. Users hold keys in the **eID Wallet**. The **Provisioner** creates their eVault. The **Registry** resolves W3IDs to eVault URLs and hosts the platform directory. The **Ontology** service publishes JSON Schemas that platforms map their local schemas to. A **Web3 Adapter** on each platform bridges the local DB to the owner's eVault. When data changes, eVault fires the **Awareness Protocol** to notify every other registered platform. + +## Components + +| Component | One line | Load this reference | +|---|---|---| +| **eVault** | GraphQL data store per W3ID, Neo4j-backed, delivers webhooks on writes | [reference/evault.md](reference/evault.md) | +| **W3ID / eName** | UUID-based persistent identifier; eName = W3ID registered in Registry | [reference/identity.md](reference/identity.md) | +| **Binding Document** | Signed MetaEnvelope tying a user to an eName (id_document, photograph, social_connection, self) | [reference/identity.md](reference/identity.md) | +| **Registry** | W3ID resolution, `/entropy` for provisioning, JWKS, platform list, key-binding certs (temporary) | [reference/registry.md](reference/registry.md) | +| **Ontology** | JSON Schema draft-07 registry served at `/schemas` and `/schemas/:id` | [reference/registry.md](reference/registry.md) | +| **Provisioner** | Creates new eVaults; exposes `POST /provision` | [reference/wallet.md](reference/wallet.md) | +| **eID Wallet** | Mobile app (Tauri/SvelteKit); holds ECDSA P-256 keys in Secure Enclave / HSM | [reference/wallet.md](reference/wallet.md) | +| **wallet-sdk** | TypeScript SDK: `provision`, `authenticate`, `syncPublicKeyToEvault`; crypto-agnostic via `CryptoAdapter` | [reference/wallet.md](reference/wallet.md) | +| **Web3 Adapter** | Bridge on each platform between local DB and eVault; `handleChange` outbound, `fromGlobal` inbound | [reference/platform.md](reference/platform.md) | +| **Awareness Protocol** | eVault → `POST /api/webhook` on every other platform after writes | [reference/protocols.md](reference/protocols.md) | +| **w3ds://auth** | Session-signing authentication flow | [reference/protocols.md](reference/protocols.md) | +| **w3ds://sign** | Session-signing for arbitrary payloads (documents, votes, references) | [reference/protocols.md](reference/protocols.md) | +| **w3ds://file** | URI scheme for file blobs; format `w3ds://file?id=@/` | [reference/protocols.md](reference/protocols.md) | +| **AaaS** | Awareness-as-a-Service — production-grade replacement for eVault's direct webhook fanout | [reference/protocols.md](reference/protocols.md) | + +## Production URLs + +| Service | URL | +|---|---| +| Provisioner | `https://provisioner.w3ds.metastate.foundation` | +| Registry | `https://registry.w3ds.metastate.foundation` | +| Ontology | `https://ontology.w3ds.metastate.foundation` | + +Source: `docs/docs/W3DS Basics/Links.md`. + +## Routing rules + +When answering a user's question, load the reference file(s) below **before** writing any code or configuration. Do not fabricate ontology IDs, GraphQL field names, mapping directives, or endpoint paths from memory — grep `docs/docs/` if a reference file doesn't answer the question. + +| User question mentions... | Load | +|---|---| +| webhook controller, mapping.json, `handleChange`, `fromGlobal`, `toGlobal`, Web3 Adapter, `ownerEnamePath`, `__date`, `__calc`, `__file`, "how do I build a platform" | [reference/platform.md](reference/platform.md) | +| GraphQL, `createMetaEnvelope`, `updateMetaEnvelope`, `removeMetaEnvelope`, `bulkCreateMetaEnvelopes`, `uploadFile`, `metaEnvelope(id)`, `metaEnvelopes`, ACL, `X-ENAME`, `/whois`, `/logs`, MetaEnvelope, Envelope, Neo4j model | [reference/evault.md](reference/evault.md) | +| W3ID, eName, `@` format, X-ENAME header, Binding Document, id_document, photograph, social_connection, self, key rotation, friend-based recovery | [reference/identity.md](reference/identity.md) | +| Registry, `/resolve`, `/entropy`, `/list`, JWKS, key binding certificate, ontology ID for User / Post / Group / Ledger / Currency / Account / Binding / File, `/schemas` | [reference/registry.md](reference/registry.md) | +| w3ds://auth, w3ds://sign, Awareness Protocol packet, signature verification, ECDSA P-256, multibase, base58btc, base64 signature format, `verifySignature`, AaaS, w3ds://file URI, dereferencing files | [reference/protocols.md](reference/protocols.md) | +| eID Wallet, wallet-sdk, `provision`, `authenticate`, `syncPublicKeyToEvault`, `CryptoAdapter`, hardware vs software keys, `PATCH /public-key`, key delegation across devices | [reference/wallet.md](reference/wallet.md) | +| `pnpm dev:core`, Dev Sandbox, ports (4321 / 3001 / 4000 / 8080), `REGISTRY_ENTROPY_KEY_JWK`, `pnpm generate-entropy-jwk`, "webhook not firing", "signature verification fails", "duplicate entities" | [reference/dev-setup.md](reference/dev-setup.md) | + +If the question spans multiple topics (common for platform builds), load the two or three most relevant references in one turn rather than piecemeal. + +## Do not guess + +The docs are ground truth. Any of these values, if guessed, is almost certainly wrong: + +- **Ontology UUIDs** — memorized table lives in [reference/registry.md](reference/registry.md). `w3ds-file-v1` is a **string literal**, not a UUID. +- **GraphQL field / mutation names** — `createMetaEnvelope` is the idiomatic name; `storeMetaEnvelope` is a legacy alias still used internally by the Web3 Adapter's `EVaultClient`. Full signatures live in [reference/evault.md](reference/evault.md). +- **Mapping directive syntax** — `__date(...)`, `__calc(...)`, `__file(...)`, `tableName(path),globalAlias`, and array `users(participants[].id),participantIds` — verbatim examples in [reference/platform.md](reference/platform.md). +- **Signature encoding** — software keys emit base64 raw 64-byte (r || s); hardware keys emit multibase base58btc (`z...`). See [reference/protocols.md](reference/protocols.md). +- **Endpoint paths and headers** — every eVault request needs `X-ENAME: @`. The `/provision` endpoint lives on the Provisioner, not eVault-core (though in local dev they run in the same eVault-core process on port 3001). + +If uncertain, `grep -r docs/docs/` before writing the answer. + +## Terminology anchors + +Common confusion points — internalize these once: + +- **MetaEnvelope vs Envelope**: MetaEnvelope is the top-level entity (one post, one user). Envelope is a single field of that entity, stored as its own Neo4j node linked via `LINKS_TO`. +- **W3ID vs eName**: All eNames are W3IDs. Only W3IDs registered in the Registry are eNames (resolvable). Both use the `@` format when global. +- **Ontology vs schema**: "Ontology" in this ecosystem refers to a specific JSON Schema published by the Ontology service and referenced by its schemaId (a W3ID). Do not confuse with generic "ontology" from semantic web. +- **Platform vs post-platform**: A platform participates in W3DS via a Web3 Adapter and a `/api/webhook` endpoint. A post-platform is a platform that operates in "dataless" mode — it doesn't own the data, users' eVaults do. +- **`w3ds-file-v1` vs `File` ontology**: `w3ds-file-v1` is the low-level storage envelope created by `uploadFile` for blob dereferencing. The `File` ontology (`a1b2c3d4-e5f6-7890-abcd-ef1234567890`) is a higher-level platform record for file-manager / esigner style apps. They are not interchangeable — different field names, different layer. Detail in [reference/protocols.md](reference/protocols.md). +- **Awareness Protocol vs AaaS**: Awareness Protocol is the prototype-level fire-and-forget fanout from eVault-core. Awareness-as-a-Service (AaaS) is the production-grade replacement with subscriptions, persistence, retries, and a dead-letter queue. +- **`storeMetaEnvelope` / `updateMetaEnvelopeById`**: Legacy GraphQL mutation names, still used internally by the Web3 Adapter's `EVaultClient`. External integrations should use `createMetaEnvelope` / `updateMetaEnvelope` / `removeMetaEnvelope` instead. + +## Working style + +- Always resolve the eVault URL for a user via the Registry before hitting `/graphql` or `/whois`. Do not hardcode eVault URLs. +- Every GraphQL and HTTP call to eVault needs `X-ENAME`. Missing this header is the most common cause of 400s. +- ACLs in the prototype are all-or-nothing except for `["*"]`. There is no read-only-without-write yet. +- Webhook delivery is fire-and-forget and prototype-level: no retries, no ordering, no at-least-once. Design your platform's webhook controller to be **idempotent** on global `id`. +- After `storeMetaEnvelope` there is a 3-second delay before webhook fanout to prevent ping-pong. `updateMetaEnvelopeById` fanout is immediate. +- If the user is running things locally, refer them to [reference/dev-setup.md](reference/dev-setup.md) before troubleshooting — most sync bugs come from a service that isn't running or a missing env var. diff --git a/skills/w3ds/reference/dev-setup.md b/skills/w3ds/reference/dev-setup.md new file mode 100644 index 000000000..d63b795ea --- /dev/null +++ b/skills/w3ds/reference/dev-setup.md @@ -0,0 +1,227 @@ +# Local dev + debugging + +One command spins up the full W3DS core stack. Most sync bugs come from a service that isn't running or a missing env var — always verify the stack is healthy before hunting deeper. Source: `docs/docs/Post Platform Guide/local-dev-quick-start.md`, `dev-sandbox.md`. + +## Prerequisites + +- **Docker** — for Postgres and Neo4j. +- **Node.js 18+** and **pnpm**. +- **`.env`** in the repo root (copy from `.env.example` if present). + +## Environment + +Minimum `.env` for the core stack: + +```bash +# Postgres (used by registry) +POSTGRES_USER=postgres +POSTGRES_PASSWORD=postgres +REGISTRY_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/registry + +# Registry: ES256 key for signing entropy tokens (REQUIRED) +REGISTRY_ENTROPY_KEY_JWK='' + +# Neo4j (used by evault-core) +NEO4J_USER=neo4j +NEO4J_PASSWORD=your-password +NEO4J_URI=bolt://127.0.0.1:7687 + +# So sandbox and evault-core can talk to registry/provisioner +PUBLIC_REGISTRY_URL=http://localhost:4321 +PUBLIC_PROVISIONER_URL=http://localhost:3001 +PUBLIC_EVAULT_SERVER_URI=http://localhost:4000 +REGISTRY_SHARED_SECRET=dev-secret-change-me +``` + +### Generating `REGISTRY_ENTROPY_KEY_JWK` + +The Registry signs entropy tokens with an ES256 key. Generate one: + +```bash +pnpm generate-entropy-jwk +``` + +Paste the output into `.env` as `REGISTRY_ENTROPY_KEY_JWK=''`. Keep this key private. Reuse the same value across local dev if you need tokens to verify elsewhere. + +## One-command bootstrap + +```bash +pnpm install +pnpm dev:core +``` + +That starts: + +1. **Postgres** (5432) and **Neo4j** (7474 HTTP, 7687 Bolt) via `docker-compose.databases.yml`. +2. Waits for Postgres to be ready. +3. Runs registry + eVault-core migrations. +4. Starts **registry** (4321), **eVault-core** (3001 provisioning + 4000 GraphQL), and **dev-sandbox** (8080) in parallel. + +Stop with Ctrl+C. Stop only databases with `pnpm docker:core:down`. + +### Or step-by-step + +```bash +pnpm dev:core:docker # start Postgres + Neo4j +pnpm dev:core:wait # wait for 5432 and 7687 +pnpm dev:core:migrate # migrations +pnpm dev:core:apps # start registry, eVault-core, dev-sandbox +``` + +## Ports + +| Service | Port | Notes | +|---|---|---| +| Postgres | 5432 | Registry DB | +| Neo4j HTTP | 7474 | Browser UI | +| Neo4j Bolt | 7687 | eVault-core connection | +| Registry | 4321 | `/resolve`, `/entropy`, `/list`, `/.well-known/jwks.json` | +| eVault-core (provisioning) | 3001 | `POST /provision` (also acts as the Provisioner in local dev) | +| eVault-core (GraphQL) | 4000 | `/graphql`, `/whois`, `/logs`, `/files/:id`, `PATCH /public-key` | +| **Dev Sandbox** | **8080** | Browser wallet substitute | + +Open **http://localhost:8080** for the Dev Sandbox. + +## Dev Sandbox — the wallet substitute + +Source: `docs/docs/Post Platform Guide/dev-sandbox.md`. + +The Dev Sandbox is a minimal browser app that uses `wallet-sdk` with a Web Crypto adapter. It lets you: + +- **Provision** — generate a key pair, get entropy, call the Provisioner. On success, syncs the public key to the eVault and creates a random UserProfile automatically. Result: usable `w3id` + `evaultUri`. +- **Identities** — stored in browser localStorage. Select one to use for auth / sign. +- **Paste any `w3ds://auth` or `w3ds://sign` URI** → click **Perform** to sign the session and POST to the callback URL. +- **Sign payload** — arbitrary string signing for custom flows. +- **Log panel** — split-screen debug log. + +### Running standalone + +If Registry and eVault-core are already up (via Docker or another terminal): + +```bash +pnpm --filter dev-sandbox dev +``` + +Reads `PUBLIC_REGISTRY_URL` and `PUBLIC_PROVISIONER_URL` from the repo root `.env`. Point these at your target stack (local, staging). + +### Testing a platform's auth flow with the sandbox + +1. Start your platform. Ensure it exposes `GET /api/auth/offer` and `POST /api/auth`. +2. Start the sandbox (`pnpm dev:core` or `pnpm --filter dev-sandbox dev`), open http://localhost:8080. +3. Click "Provision new eVault". Wait for "Public key synced" and "UserProfile created" in the log. +4. Get an auth offer from your platform (open the login page or curl the offer endpoint) → copy the `w3ds://auth?...` URL. +5. Paste into the sandbox's "Paste any w3ds URI" field → click **Perform**. The sandbox signs the session and POSTs to your callback. +6. Verify your platform received the POST, verified the signature, and issued a session. + +Same pattern for `w3ds://sign` — paste the URI, click Perform, watch your callback receive `{ sessionId, signature, w3id, message }`. + +## Debugging playbook + +### Webhook not firing on other platforms + +**Symptom**: platform A writes to its eVault, platform B never gets a webhook. + +Check in order: + +1. Is your platform registered? Query `GET http://localhost:4321/list` and confirm your platform's URL is in the response. +2. If the write is a **create** (not update), remember there is a **3-second delay** before fanout. Wait, then re-check. +3. Is your `/api/webhook` endpoint publicly reachable from eVault-core? (In local dev, `localhost` works. In containers, use the service name or host.docker.internal.) +4. Does the packet's `schemaId` match a mapping in your Web3 Adapter? If not, your controller correctly drops it — that's expected. + +### Duplicate entities on sync + +**Symptom**: writing to platform A causes duplicates to appear on platform A after the webhook echo (or on other platforms after their echoes). + +Cause: ID mapping not persisted. Check that `mappingDb.storeMapping({ localId, globalId })` runs after every successful create in both directions: + +- Outbound: after `EVaultClient.storeMetaEnvelope` returns the new `globalId`. +- Inbound: after your webhook controller creates the local entity. + +Verify with a query to your `MappingDatabase`: for a known global ID, `getLocalId(globalId)` must return the local row's ID. + +### Signature verification fails + +Common causes, in order of frequency: + +1. **Wrong `payload`** — the `session` field must be the exact string that was signed, byte-for-byte. If your client is re-serializing JSON, whitespace differences will break verification. +2. **Signature encoding mismatch** — hardware-key signatures use multibase base58btc (starts with `z`). Software-key signatures use plain base64. The validator auto-detects, but if you extract signature bytes yourself, you can slip up. +3. **Key never synced to eVault** — call `GET {evaultUri}/whois -H "X-ENAME: @user"`. Empty `keyBindingCertificates` means the wallet's `PATCH /public-key` never ran. Re-run onboarding or use `wallet-sdk`'s `syncPublicKeyToEvault`. +4. **Certificate expired** — key-binding certs are valid 1 hour. If the eVault has been idle, the cert may have expired. eVault regenerates on demand — retry. +5. **JWKS endpoint unreachable** — `GET http://localhost:4321/.well-known/jwks.json` should return a JWK set. If it 404s, the Registry is misconfigured (usually a missing `REGISTRY_ENTROPY_KEY_JWK` env var). + +### `/whois` returns empty certificates + +Wallet has never called `PATCH /public-key` for this eName. Fix by: + +- Re-running the onboarding provision (with `publicKey` in the body), or +- Calling `syncPublicKeyToEvault` from wallet-sdk explicitly. + +If the wallet's `authToken` for `PATCH /public-key` is misconfigured, the endpoint returns 401 — check `PUBLIC_EID_WALLET_TOKEN` in the sandbox / wallet env. + +### Neo4j "encryption setting" or connection refused + +The stack ships Neo4j 4.4 (unencrypted Bolt by default). If a previous 5.x run left volumes, purge and recreate: + +```bash +docker compose -f docker-compose.databases.yml down +docker volume rm metastate_neo4j_data 2>/dev/null || true +docker compose -f docker-compose.databases.yml up -d +pnpm dev:core +``` + +Otherwise ensure `.env` has: + +```bash +NEO4J_URI=bolt://127.0.0.1:7687 +NEO4J_USER=neo4j +NEO4J_PASSWORD=your-password +``` + +### Provision returns 400 + +Check request body order: + +1. `registryEntropy` must be a fresh JWT from `GET /entropy` (valid 1 hour). +2. `namespace` should be a UUID. +3. `verificationId` — in local dev, use the demo verification code your Provisioner is configured to accept (see the Provisioner's env). +4. `publicKey` — optional; if provided, must be multibase-encoded ECDSA P-256 SPKI DER. + +### Auth POST rejects "User not found" + +Order of operations matters. A user must exist in your platform's DB before they can log in — the User is created by your webhook controller when the User MetaEnvelope (ontology `550e8400-...440000`) arrives. + +Correct sequence for a new user: + +1. User provisions their eVault (via wallet or sandbox). +2. Wallet / sandbox writes a User MetaEnvelope to that eVault. +3. eVault fires webhooks including yours. +4. Your webhook controller creates the local User row. +5. Now the user can log in. + +If step 3 or 4 didn't happen, `/api/auth` correctly returns 404. + +### `pnpm dev:core` starts but sandbox can't provision + +The sandbox reads `PUBLIC_REGISTRY_URL` and `PUBLIC_PROVISIONER_URL` from `.env` at build time. If you edited `.env` after starting, restart the sandbox process (`pnpm --filter dev-sandbox dev`). + +## Databases-only mode + +If you run app services yourself and only need Postgres + Neo4j: + +```bash +pnpm docker:core +# or +docker compose -f docker-compose.databases.yml up -d +``` + +Stop: + +```bash +pnpm docker:core:down +``` + +## References in the docs + +- Local dev quick start: `docs/docs/Post Platform Guide/local-dev-quick-start.md` +- Dev Sandbox: `docs/docs/Post Platform Guide/dev-sandbox.md` +- Full Docker setup + platforms: repo root `README.md` diff --git a/skills/w3ds/reference/evault.md b/skills/w3ds/reference/evault.md new file mode 100644 index 000000000..05b29eb54 --- /dev/null +++ b/skills/w3ds/reference/evault.md @@ -0,0 +1,342 @@ +# eVault — data store + GraphQL + +The eVault is the personal data store for a single W3ID. One eVault per tenant, Neo4j-backed, GraphQL at `/graphql`, HTTP endpoints for identity/log/file resolution. Source: `docs/docs/Infrastructure/eVault.md`. + +## Data model + +- **MetaEnvelope**: top-level container for one entity (post, user, message). Fields: `id` (W3ID), `ontology` (schemaId W3ID), `acl` (array), `envelopes` (list). +- **Envelope**: one field of a MetaEnvelope stored as its own Neo4j node. Fields: `id`, `fieldKey` (e.g. `"content"`, `"authorId"`), `ontology` (legacy alias for `fieldKey`), `value`, `valueType` (`"string" | "number" | "object" | "array"`). +- Neo4j structure: `(MetaEnvelope {id, ontology, acl}) -[:LINKS_TO]-> (Envelope {id, fieldKey, value, valueType})`. +- Flat graph, not nested — enables field-level updates and searching, at the cost of reconstruction complexity for deeply nested payloads. + +## Required header + +Every GraphQL and HTTP call to eVault MUST include: + +```http +X-ENAME: @ +``` + +Missing this header returns 400 or "access denied" — it is the #1 integration bug. + +## GraphQL — idiomatic API + +All shown below verified against `docs/docs/Infrastructure/eVault.md`. Endpoint: `POST {evaultUrl}/graphql`. + +### Query one + +```graphql +query { + metaEnvelope(id: "global-id-123") { + id + ontology + parsed + envelopes { id fieldKey value valueType } + } +} +``` + +`parsed` returns the reconstructed object form (payload dict). Prefer it over walking envelopes yourself. + +### Query many (cursor-paginated, filterable) + +```graphql +query { + metaEnvelopes( + filter: { + ontologyId: "550e8400-e29b-41d4-a716-446655440001" + search: { term: "hello", caseSensitive: false, mode: CONTAINS } + } + first: 10 + after: "cursor-string" + ) { + edges { cursor node { id ontology parsed } } + pageInfo { hasNextPage hasPreviousPage startCursor endCursor } + totalCount + } +} +``` + +Filter fields: `ontologyId`, `search.term`, `search.caseSensitive`, `search.fields` (array of fieldKeys to restrict search to), `search.mode` (`CONTAINS | STARTS_WITH | EXACT`). Pagination: `first`/`after` forward, `last`/`before` backward. + +### Create + +```graphql +mutation { + createMetaEnvelope(input: { + ontology: "550e8400-e29b-41d4-a716-446655440001" + payload: { + content: "Hello, world!" + mediaUrls: [] + authorId: "@e4d909c2-..." + createdAt: "2025-01-24T10:00:00Z" + } + acl: ["*"] + }) { + metaEnvelope { id ontology parsed envelopes { id fieldKey value } } + errors { field message code } + } +} +``` + +Structured payload response — always check `errors[]` even on 200 OK. + +### Update + +```graphql +mutation { + updateMetaEnvelope( + id: "global-id-123" + input: { + ontology: "550e8400-e29b-41d4-a716-446655440001" + payload: { content: "Updated content", mediaUrls: [] } + acl: ["*"] + } + ) { + metaEnvelope { id ontology parsed } + errors { message code } + } +} +``` + +### Remove + +```graphql +mutation { + removeMetaEnvelope(id: "global-id-123") { + deletedId + success + errors { message code } + } +} +``` + +### Bulk create + +For migrations and initial seeds only. Requires `Authorization: Bearer ` in addition to `X-ENAME`. Supports optional `id` per input (preserves IDs across migrations). + +```graphql +mutation { + bulkCreateMetaEnvelopes( + inputs: [ + { id: "custom-id-1", ontology: "550e8400-...", payload: {...}, acl: ["*"] } + { ontology: "550e8400-...", payload: {...}, acl: ["@platform-a.w3id"] } + ] + skipWebhooks: false + ) { + results { id success error } + successCount + errorCount + errors { message code } + } +} +``` + +`skipWebhooks: true` only takes effect for migration-authorized platforms (e.g. Emover). Regular platform tokens ignore it. + +### File upload + +```graphql +mutation UploadFile($input: UploadFileInput!) { + uploadFile(input: $input) { + uri # w3ds://file?id=@/ + metaEnvelopeId + publicUrl # direct object-storage URL + errors { field message code } + } +} +``` + +`UploadFileInput`: `filename` (string), `contentType` (string, MIME), `content` (base64 or `data:` URI), `acl` (array). Decoded size must be ≤ 50 MB. Requires `X-ENAME` and object storage configured on the eVault. Detail on the `w3ds://file` scheme → [protocols.md](protocols.md). + +### Binding documents + +```graphql +query { + bindingDocument(id: "meta-envelope-id") { + subject type data + signatures { signer signature timestamp } + } +} + +query { + bindingDocuments(type: id_document, first: 10) { + edges { node { subject type data signatures { signer signature timestamp } } } + pageInfo { hasNextPage endCursor } + totalCount + } +} + +mutation { + createBindingDocument(input: { + subject: "@e4d909c2-..." + type: id_document # or photograph | social_connection | self + data: { vendor: "onfido", reference: "ref-12345", name: "John Doe" } + ownerSignature: { + signer: "@e4d909c2-..." + signature: "sig_abc123..." + timestamp: "2025-01-24T10:00:00Z" + } + }) { + metaEnvelopeId + bindingDocument { subject type data signatures { signer signature timestamp } } + errors { message code } + } +} + +mutation { + createBindingDocumentSignature(input: { + bindingDocumentId: "meta-envelope-id" + signature: { signer: "@counterparty-uuid", signature: "sig_xyz...", timestamp: "2025-01-24T11:00:00Z" } + }) { + bindingDocument { subject type signatures { signer signature timestamp } } + errors { message code } + } +} +``` + +Binding documents are stored as MetaEnvelopes with ontology `b1d0a8c3-4e5f-6789-0abc-def012345678`. The MetaEnvelope ID is the binding document ID. See [identity.md](identity.md) for the type-specific data shapes. + +## GraphQL — legacy names (still valid) + +Preserved for backward compat; internal use by the Web3 Adapter's `EVaultClient`: + +- `storeMetaEnvelope(input: MetaEnvelopeInput!)` → use `createMetaEnvelope` +- `updateMetaEnvelopeById(id: String!, input: MetaEnvelopeInput!)` → use `updateMetaEnvelope` +- `deleteMetaEnvelope(id: String!)` → use `removeMetaEnvelope` (legacy returned `Boolean!`; new returns a payload) +- `getMetaEnvelopeById(id: String!)` → use `metaEnvelope(id: ID!)` +- `findMetaEnvelopesByOntology(ontology: String!)` → use `metaEnvelopes(filter: { ontologyId: ... })` +- `searchMetaEnvelopes(ontology: String!, term: String!)` → use `metaEnvelopes(filter: { search: ... })` +- `updateEnvelopeValue(envelopeId: String!, newValue: JSON!)` — field-level update, no idiomatic replacement + +If you see `storeMetaEnvelope` in Web3 Adapter code, that is the internal method name on `EVaultClient` and is correct in that context. + +## HTTP endpoints + +### GET /whois + +```bash +curl http://localhost:4000/whois -H "X-ENAME: @user-a.w3id" +``` + +Returns: + +```json +{ + "w3id": "@user-a.w3id", + "evaultId": "@evault-identifier", + "keyBindingCertificates": ["eyJhbGciOiJFUzI1NiIs...", "..."] +} +``` + +Certificates are ES256 JWTs signed by the Registry. Payload: `{ ename, publicKey, exp, iat }`. Valid 1 hour. Verify signature verification recipe in [protocols.md](protocols.md). + +### GET /logs + +Paginated envelope operation log. Query params: `limit` (default 20, max 100), `cursor`. + +```bash +curl "http://localhost:4000/logs?limit=20" -H "X-ENAME: @user-a.w3id" +``` + +Response: + +```json +{ + "logs": [ + { + "id": "log-entry-id", + "eName": "@user-a.w3id", + "metaEnvelopeId": "meta-envelope-id", + "envelopeHash": "sha256-hex", + "operation": "create", // create | update | delete | update_envelope_value + "platform": "https://platform.example.com", + "timestamp": "2025-02-04T12:00:00.000Z", + "ontology": "550e8400-e29b-41d4-a716-446655440001" + } + ], + "nextCursor": "2025-02-04T12:00:00.000Z|log-entry-id", + "hasMore": true +} +``` + +URL-encode the cursor when following pagination — it contains `|`. + +### GET /files/:metaEnvelopeId + +Dereferences a `w3ds://file` URI. Returns a **302 redirect** to the public object-storage URL. Requires `X-ENAME`. See [protocols.md](protocols.md) for the URI scheme. + +### PATCH /public-key + +Wallet endpoint for key sync. Body: `{ publicKey }`. Headers: `X-ENAME` (required), `Authorization: Bearer ` (required). eVault stores the key and requests a fresh key-binding certificate from the Registry. See [wallet.md](wallet.md). + +## Access control + +ACLs are string arrays on each MetaEnvelope: + +- `["*"]` — anyone can read; only the eVault owner can write. +- `["@user-a.w3id"]` — user A can read AND write. +- `["@user-a.w3id", "@user-b.w3id"]` — both can read and write. + +Prototype limitation: no read-only-without-write except for `["*"]`. Fine-grained perms are on the roadmap. + +Access enforcement flow: + +1. Extract W3ID from `X-ENAME` header or Bearer token. +2. Check requester's W3ID is in the ACL. +3. Strip the ACL field from the response (security). +4. Grant or deny. + +Special cases: + +- `storeMetaEnvelope` (legacy `createMetaEnvelope` alias): requires only `X-ENAME`, no Bearer token. +- `["*"]`: any authenticated request can read. +- Bulk create requires a Bearer token in addition to `X-ENAME`. + +## Webhook delivery (Awareness Protocol) + +After a `createMetaEnvelope` (or legacy `storeMetaEnvelope`), eVault: + +1. Persists to Neo4j. +2. Waits **3 seconds** (create only — `updateMetaEnvelope` fires immediately). +3. `GET /platforms` on the Registry → list of platform base URLs. +4. Filters out the requesting platform (identified from the Bearer token's `platform` claim, URL-normalized). +5. `POST /api/webhook` on every remaining platform in parallel. 5s timeout per call. No retries. Fire-and-forget. + +Payload: + +```json +{ + "id": "a1b2c3d4-...", + "w3id": "@user-a.w3id", + "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "data": { + "content": "Hello, world!", + "mediaUrls": [], + "authorId": "@e4d909c2-...", + "createdAt": "2025-01-24T10:00:00Z" + }, + "evaultPublicKey": "z..." +} +``` + +For the receiving side (writing a `/api/webhook` handler), see [platform.md](platform.md). + +## Key binding certificates + +- Stored in the eVault, retrieved via `/whois`. +- Issued by the Registry as ES256 JWTs. Payload: `{ ename, publicKey, exp, iat }`. TTL 1 hour. +- Purpose: (a) tamper protection over the wire, (b) Registry accountability for W3ID↔publicKey binding. +- Lifecycle: created during eVault provisioning if `publicKey` was included; refreshed on `PATCH /public-key`. +- Multi-device: one certificate per key. Verifier iterates and returns success on the first match. + +## Multi-tenancy + +The Provisioner supports multiple W3IDs sharing infrastructure, but each eVault instance is dedicated to a single tenant. Database queries are always filtered by W3ID; there is no cross-tenant read except through ACLs. + +## References in the docs + +- Full spec: `docs/docs/Infrastructure/eVault.md` +- Data model + ontology field semantics: `docs/docs/Infrastructure/Ontology.md` +- Webhook packet + delivery mechanics: `docs/docs/W3DS Protocol/Awareness-Protocol.md` +- Key binding certificate detail: `docs/docs/Infrastructure/eVault-Key-Delegation.md` diff --git a/skills/w3ds/reference/identity.md b/skills/w3ds/reference/identity.md new file mode 100644 index 000000000..76df25843 --- /dev/null +++ b/skills/w3ds/reference/identity.md @@ -0,0 +1,125 @@ +# Identity — W3ID, eName, Binding Documents + +W3IDs identify every user, group, eVault, and MetaEnvelope in the ecosystem. An eName is a W3ID that has been registered in the Registry and is therefore resolvable to a service URL. Source: `docs/docs/W3DS Basics/W3ID.md`, `eName.md`, `Binding-Documents.md`. + +## W3ID + +UUID-based (RFC 4122), persistent, globally unique. Two forms: + +| Form | Format | Example | Use | +|---|---|---|---| +| **Global** (eName) | `@` | `@e4d909c2-5d2f-4a7d-9473-b34b6c0f1a5a` | Cross-platform identity, ACLs, X-ENAME header | +| **Local** | plain `` | `f2a6743e-8d5b-43bc-a9f0-1c7a3b9e90d7` | Object identifier within one eVault | + +Namespace has range 2^122 (from UUID); collision probability is negligible. Global IDs are case-insensitive. + +Key property: **loosely bound to keys**. The W3ID is not derived from any key, which is what enables: + +- **Key rotation** without changing identity (key compromise, device loss). +- **Friend-based recovery** — a trust list (2–3 friends or notaries) can approve key changes. +- **eVault migration** — the Registry can hold also-known-as / redirect records so an old W3ID still resolves after a move. + +## eName vs W3ID + +Every eName is a W3ID. Only W3IDs registered in the Registry are eNames. + +| | W3ID (unregistered) | eName | +|---|---|---| +| Format | `@` | `@` | +| Resolvable via Registry | No | Yes | +| Used in `X-ENAME` header | No | Yes | +| Primary role | Local identifier | Cross-platform identity | + +If you're building a platform, users and groups you interact with will always have eNames — never bare W3IDs. + +## X-ENAME header + +Required on every eVault GraphQL / HTTP request: + +```http +X-ENAME: @e4d909c2-5d2f-4a7d-9473-b34b6c0f1a5a +``` + +Determines: which eVault to route the request to, ACL enforcement, log ownership. Missing header = 400. + +## Where W3IDs / eNames appear + +- **Users, groups**: each has a persistent eName that anchors keys and (via binding documents) physical identity. +- **eVaults**: an eVault has its own internal W3ID (used for clone sync). The owner's eName identifies the "owner" for ACL / whois purposes. +- **MetaEnvelopes**: `id` is a W3ID. Ownership is by the eVault whose owner-eName was in `X-ENAME` at creation time. +- **ACLs**: arrays of eNames (or `["*"]`). See [evault.md](evault.md#access-control). +- **Key binding certificates**: JWTs whose payload binds an eName to a public key. + +## Binding Documents + +A Binding Document is a special MetaEnvelope (ontology `b1d0a8c3-4e5f-6789-0abc-def012345678`) that ties a subject eName to a real-world credential or claim. Every binding document has: + +- `subject` — the eName being bound (with `@` prefix) +- `type` — one of `id_document | photograph | social_connection | self` +- `data` — type-specific payload +- `signatures[]` — at least the owner's signature; counterparty signatures appendable + +Signature shape: + +```typescript +interface BindingDocumentSignature { + signer: string; // eName or keyID of who signed + signature: string; // Cryptographic signature (base64 or multibase) + timestamp: string; // ISO 8601 +} +``` + +### Types + data shapes + +**id_document** — binds eName to a KYC-verified ID document: + +```json +{ "vendor": "onfido", "reference": "ref-12345", "name": "John Doe" } +``` + +**photograph** — binds eName to a selfie / profile photo: + +```json +{ "photoBlob": "base64encodedimage==" } +``` + +**social_connection** — binds two eNames to a claimed relationship: + +```json +{ + "kind": "social_connection", + "name": "Alice Smith", + "parties": ["@ename-1", "@ename-2"], + "relation_description": "Known each other since university" +} +``` + +`parties` must be exactly two eNames. `kind` is a required discriminator. + +**self** — user's self-declared identity: + +```json +{ "name": "Bob Jones" } +``` + +### Storage + +- MetaEnvelope ID = binding document ID (no separate id field on the document itself). +- ACL is restricted to the subject's eName by default. +- Stored in the subject's own eVault (their eName owns the MetaEnvelope). + +### GraphQL operations + +See the full mutations and queries in [evault.md § Binding documents](evault.md#binding-documents). Key idea: use `createBindingDocument` to create with the owner signature, then `createBindingDocumentSignature` to append counterparty signatures for chain-of-trust verification. + +## Document binding to physical identity + +The W3ID system supports binding an identity to a passport or other physical document via a certified binding document. Passport verification itself is out of scope for W3ID — it's handled by the eID Wallet's verification integrations. The binding is loose (entropy derived from passport details, certified by a root CA) so the identity survives passport renewal. + +## References in the docs + +- W3ID spec: `docs/docs/W3DS Basics/W3ID.md` +- eName vs W3ID: `docs/docs/W3DS Basics/eName.md` +- Binding document types + operations: `docs/docs/W3DS Basics/Binding-Documents.md` +- ACL semantics: `docs/docs/Infrastructure/eVault.md` (§ Access Control) +- Key binding certificates: `docs/docs/Infrastructure/eVault-Key-Delegation.md` diff --git a/skills/w3ds/reference/platform.md b/skills/w3ds/reference/platform.md new file mode 100644 index 000000000..d133242b9 --- /dev/null +++ b/skills/w3ds/reference/platform.md @@ -0,0 +1,387 @@ +# Building a post-platform + +This is the primary developer reference. A platform participating in W3DS needs four things: an auth flow, a webhook endpoint, JSON mapping files, and a Web3 Adapter wired to the local DB. Source: `docs/docs/Post Platform Guide/*.md`, `docs/docs/Infrastructure/Web3-Adapter.md`. + +## The four required pieces + +| Piece | What | Reference doc | +|---|---|---| +| **Auth endpoints** | `GET /api/auth/offer` + `POST /api/auth`, using `signature-validator` | `docs/docs/Post Platform Guide/getting-started.md` | +| **Webhook endpoint** | `POST /api/webhook` — idempotent, uses `adapter.fromGlobal` + mapping DB | `docs/docs/Post Platform Guide/webhook-controller.md` | +| **Mapping files** | JSON per local table describing the global schema mapping | `docs/docs/Post Platform Guide/mapping-rules.md` | +| **Web3 Adapter** | Instance holding the mapping configs, mapping DB, and eVault client; call `handleChange(...)` after every DB write | `docs/docs/Infrastructure/Web3-Adapter.md` | + +If your app is stateless — writes directly to eVaults and doesn't own a local DB — you can skip the Web3 Adapter entirely. Adapter is only needed when a platform DB has to stay in sync with eVaults. + +## Auth flow + +### `GET /api/auth/offer` + +Generate a session, build a `w3ds://auth` URI, return it (and the sessionId if your client needs it): + +```typescript +getOffer = async (req: Request, res: Response) => { + const url = new URL("/api/auth", process.env.PUBLIC_BASE_URL).toString(); + const sessionId = uuidv4(); + const offer = `w3ds://auth?redirect=${url}&session=${sessionId}&platform=YOUR_PLATFORM`; + res.json({ offer, sessionId }); +}; +``` + +Persist `sessionId` with a 5-minute TTL — reject reuse to prevent replays. + +### `POST /api/auth` + +The wallet POSTs the signed session here: + +```typescript +import { verifySignature } from "signature-validator"; +import { signToken } from "../utils/jwt"; + +login = async (req: Request, res: Response) => { + const { w3id, session, signature, appVersion } = req.body; + + if (!w3id || !session || !signature) { + return res.status(400).json({ error: "Missing required fields" }); + } + + const verificationResult = await verifySignature({ + eName: w3id, + signature, + payload: session, + registryBaseUrl: process.env.PUBLIC_REGISTRY_URL, + }); + + if (!verificationResult.valid) { + return res.status(401).json({ error: "Invalid signature", message: verificationResult.error }); + } + + // Users must exist before login. They are created by your webhook handler + // when the User ontology (550e8400-...440000) MetaEnvelope arrives. + const user = await this.userService.findByEname(w3id); + if (!user) { + return res.status(404).json({ error: "User not found", message: "User must be created via eVault webhook before authentication" }); + } + + const token = signToken({ userId: user.id }); + res.status(200).json({ user, token }); +}; +``` + +Guard protected routes with `authMiddleware` + `authGuard`: + +```typescript +export const authMiddleware = async (req, res, next) => { + const authHeader = req.headers.authorization; + if (!authHeader?.startsWith('Bearer ')) return next(); + try { + const { userId } = verifyToken(authHeader.substring(7)); + const user = await userService.getUserById(userId); + if (user) req.user = user; + } catch { /* invalid token — continue unauthenticated */ } + next(); +}; + +export const authGuard = (req, res, next) => { + if (!req.user) return res.status(401).json({ error: "Unauthorized" }); + next(); +}; +``` + +Env vars: `JWT_SECRET`, `PUBLIC_REGISTRY_URL`. Detail in [protocols.md § w3ds://auth](protocols.md#w3dsauth-authentication). + +## Webhook controller + +`POST /api/webhook`. Receives Awareness Protocol packets from every eVault whenever data changes anywhere. Contract: + +1. Find the mapping by `schemaId`. +2. `fromGlobal(...)` → local shape. +3. Look up (or create) the local ID for the packet's global `id`. +4. Persist the mapping. +5. Return 200. + +Reference implementation (adapted from eCurrency-api): + +```typescript +handleWebhook = async (req: Request, res: Response) => { + const globalId = req.body.id; + const schemaId = req.body.schemaId; + + try { + const mapping = Object.values(this.adapter.mapping).find( + (m: any) => m.schemaId === schemaId, + ); + if (!mapping) throw new Error("No mapping found"); + + const local = await this.adapter.fromGlobal({ data: req.body.data, mapping }); + + let localId = await this.adapter.mappingDb.getLocalId(globalId); + + if (mapping.tableName === "users") { + const entity = localId + ? await this.userService.updateUser(localId, local) + : await this.userService.createUser(local); + if (!localId) { + await this.adapter.mappingDb.storeMapping({ localId: entity.id, globalId }); + } + } else if (mapping.tableName === "groups") { + // ... same pattern per entity type + } + + res.status(200).send(); + } catch (e) { + console.error("Webhook error:", e); + res.status(500).send(); + } +}; +``` + +Route registration: + +```typescript +app.post("/api/webhook", webhookController.handleWebhook); // no auth on webhook +``` + +The endpoint receives packets for **every** ontology — you must filter and drop packets your platform doesn't consume. + +**Idempotency is mandatory.** Same `globalId` may arrive more than once. Never create a second local row for the same global ID; always upsert. + +Detail: `docs/docs/Post Platform Guide/webhook-controller.md`. + +## Mapping directives + +Mapping files describe how local table fields ↔ global ontology fields. Same file is used both directions: `toGlobal` for outbound sync, `fromGlobal` for inbound webhooks. + +### File shape + +```json +{ + "tableName": "local_table_name", + "schemaId": "550e8400-...", + "ownerEnamePath": "ename", + "ownedJunctionTables": ["junction_table1"], + "localToUniversalMap": { + "localField": "globalField", + "localRelation": "tableName(relationPath),globalAlias" + } +} +``` + +- `tableName` — local table / entity name. +- `schemaId` — global ontology W3ID (from [registry.md § Canonical ontology W3IDs](registry.md#canonical-ontology-w3ids)). +- `ownerEnamePath` — how to determine which eVault owns rows in this table. Supports fallbacks with `||`. +- `ownedJunctionTables` — for many-to-many relationships; when a junction row changes, the adapter re-syncs the parent. +- `readOnly` (optional) — when `true`, `handleChange` skips this table for outbound sync. +- `localToUniversalMap` — the field mapping. + +### Directives (verbatim, from `docs/docs/Post Platform Guide/mapping-rules.md`) + +**Direct field:** + +```json +"localField": "globalField" +``` + +**Relation (single):** + +```json +"localRelation": "tableName(relationPath),globalAlias" +``` + +- `tableName` — the referenced table. +- `relationPath` — path to the relation data on the local entity. +- `globalAlias` — target global field name. + +**Relation (array):** + +```json +"participants": "users(participants[].id),participantIds" +``` + +- `participants[].id` extracts `id` from each element of the local `participants` array. +- `users(...)` resolves each ID to a global user reference. +- `participantIds` is the target global field. + +**Date conversion:** + +```json +"createdAt": "__date(createdAt)" +"timestamp": "__date(calc(timestamp * 1000))" +``` + +Handles: Unix seconds (number), Firebase v8 `{_seconds}`, Firebase v9+ `{seconds}`, Firebase Timestamp objects, JS Date, UTC strings. + +**Arithmetic:** + +```json +"total": "__calc(quantity * price)" +"average": "__calc((score1 + score2 + score3) / 3)" +``` + +Supports basic ops (`+ - * /`), references other fields on the same entity, auto-resolves referenced values first. + +**File referencing (same global field name):** + +```json +"avatar": "__file(avatar)" +``` + +**File referencing (different global field name):** + +```json +"avatar": "__file(avatar),avatarUri" +``` + +Behavior: + +- The inner path (`avatar`) is the field holding the file value. +- Optional `,alias` sets the global field (defaults to the inner path). +- Value may be a **single file** or an **array**. Array paths like `__file(images[].src)` are supported. +- `toGlobal`: `data:` URI → uploaded and replaced with `w3ds://file?id=@/`. Existing `w3ds://file` URIs, plain URLs, and empty values pass through. +- `fromGlobal`: `w3ds://file` URI → dereferenced to the public object-storage URL. Other values pass through. + +Detail on the URI scheme: [protocols.md § File URIs](protocols.md#file-uris-w3dsfile). + +### `ownerEnamePath` patterns + +```json +"ownerEnamePath": "ename" // direct field on the entity +"ownerEnamePath": "users(createdBy.ename)" // nested via a relation +"ownerEnamePath": "users(participants[].ename)" // array relation +"ownerEnamePath": "users(createdBy.ename) || ename" // fallback +``` + +The adapter uses this to write to the correct owner's eVault. If it resolves to nothing, `handleChange` returns without syncing. + +### Junction tables + +Junction (many-to-many) tables that are conceptually part of a parent entity: + +```json +"ownedJunctionTables": ["user_followers", "user_following"] +``` + +When rows in a listed junction table change, the adapter re-syncs the parent entity. + +### Complete examples + +User: + +```json +{ + "tableName": "users", + "schemaId": "550e8400-e29b-41d4-a716-446655440000", + "ownerEnamePath": "ename", + "ownedJunctionTables": ["user_followers", "user_following"], + "localToUniversalMap": { + "handle": "username", + "name": "displayName", + "description": "bio", + "avatarUrl": "avatarUrl", + "ename": "ename", + "followers": "followers", + "following": "following" + } +} +``` + +Group with relations: + +```json +{ + "tableName": "groups", + "schemaId": "550e8400-e29b-41d4-a716-446655440003", + "ownerEnamePath": "users(participants[].ename)", + "localToUniversalMap": { + "name": "name", + "description": "description", + "owner": "owner", + "admins": "users(admins),admins", + "participants": "users(participants[].id),participantIds", + "createdAt": "__date(createdAt)", + "updatedAt": "__date(updatedAt)" + } +} +``` + +## Web3 Adapter — the sync engine + +Source: `docs/docs/Infrastructure/Web3-Adapter.md`. + +### Components + +- **Web3Adapter** — the main class. Config: `schemasPath`, `dbPath`, `registryUrl`, `platform`. Exposes `handleChange` and `fromGlobal`. +- **EVaultClient** — resolves eNames via Registry `/resolve`, obtains a platform token via `POST /platforms/certification`, caches clients per eName, health-checks with `HEAD /whois`. Calls `storeMetaEnvelope` / `updateMetaEnvelopeById` on the eVault (these are the internal names the client uses — externally-exposed idiomatic names are `create`/`update`). +- **Mapper** — `toGlobal({ data, mapping, mappingStore })` and `fromGlobal(...)`. +- **MappingDatabase** — SQLite store for `(local_id, global_id)`. Methods: `storeMapping`, `getLocalId(globalId)`, `getGlobalId(localId)`. If you use the bundled TypeScript adapter, ID mapping is handled for you. + +### Outbound flow (local write → eVault) + +``` +platform detects DB change + ↓ +adapter.handleChange({ data, tableName, participants }) + ↓ +lookup mapping for tableName; if missing or readOnly → return + ↓ +mappingDb.getGlobalId(localId)? + ├── yes → toGlobal(...) → EVaultClient.updateMetaEnvelopeById(globalId, ...) + └── no → toGlobal(...) → resolve ownerEname + if no owner → return + EVaultClient.storeMetaEnvelope({ w3id, data, schemaId }) + mappingDb.storeMapping({ localId, globalId }) + for each participant eName: storeReference(ownerEvault/globalId, otherEvault) + ↓ +eVault fires Awareness Protocol → other platforms' /api/webhook +``` + +The requesting platform is excluded from the fanout. Detail in [protocols.md § Awareness Protocol](protocols.md#awareness-protocol-webhooks). + +### Change detection + +The adapter does **not** poll. The platform is responsible for calling `handleChange(...)` after every write. Common patterns: + +- **ORM event listeners** (afterInsert / afterUpdate / afterDelete). +- **DB triggers** on INSERT / UPDATE / DELETE. +- **Change data capture** (CDC) on the WAL. +- **Application-level hooks** (call adapter directly after the write). +- **Transactional outbox** — write to an outbox table in the same transaction as the entity write; a worker calls `handleChange`. Best consistency guarantee. + +### Inbound flow (webhook → local write) + +Handled by the webhook controller shown earlier — the adapter's `fromGlobal` does the schema conversion. + +## Common integration bugs + +Every one of these has bitten someone. Address them in code review: + +1. **Missing `ownerEnamePath`** in a mapping → `handleChange` silently returns, entity never syncs. +2. **ID mapping not persisted** → the same entity arrives on the platform via webhook after outbound sync, and the controller creates a duplicate local row instead of recognizing the entity it just wrote. +3. **Array mapping without `[].id`** → tries to sync the raw objects instead of extracted IDs, mapper fails. +4. **Guessed ontology UUIDs** → mapping.json ships with a made-up schemaId; every webhook for that schema is silently dropped because no mapping matches. +5. **Confusing `File` ontology and `w3ds-file-v1`** → payload has the wrong field names. See [protocols.md § w3ds-file-v1 vs File ontology](protocols.md#w3ds-file-v1-vs-file-ontology--do-not-confuse). +6. **`__date` not applied on Firebase timestamps** → strings arrive instead of dates on the receiving side. +7. **Non-idempotent webhook controller** → duplicate deliveries create duplicate rows. Always upsert by global ID. +8. **Auth expects the user to exist before webhook** → for a fresh eVault, the User MetaEnvelope needs to have been synced first. Order: provision + create User in eVault → webhook creates local user → then login can succeed. +9. **Missing `X-ENAME` on adapter calls** → 400s on every eVault write. + +## Known limitations + +Prototype-level; on the roadmap: + +- Ontology versioning — `schemaId` is a single W3ID today; a `schemaVersion` key is planned. +- Conflict resolution — last-write-wins today. No merge or CRDT. +- Idempotency keys — not yet standardized on the wire. +- Transactional outbox not built-in. +- Mapping expressiveness — richer `ownerEnamePath`, better array handling planned. + +Design your platform's consistency layer accordingly. + +## References in the docs + +- Getting started (auth): `docs/docs/Post Platform Guide/getting-started.md` +- Webhook controller: `docs/docs/Post Platform Guide/webhook-controller.md` +- Mapping rules: `docs/docs/Post Platform Guide/mapping-rules.md` +- eCurrency example: `docs/docs/Post Platform Guide/ecurrency-accounts-and-ledger.md` +- Web3 Adapter architecture: `docs/docs/Infrastructure/Web3-Adapter.md` +- Awareness Protocol packet + timing: `docs/docs/W3DS Protocol/Awareness-Protocol.md` diff --git a/skills/w3ds/reference/protocols.md b/skills/w3ds/reference/protocols.md new file mode 100644 index 000000000..ef0868c66 --- /dev/null +++ b/skills/w3ds/reference/protocols.md @@ -0,0 +1,330 @@ +# W3DS protocols + +Four wire-level protocols to know: `w3ds://auth`, `w3ds://sign`, the Awareness Protocol (webhooks), and `w3ds://file` URIs. Plus signature verification, which is shared by auth and sign. Source: `docs/docs/W3DS Protocol/*.md`. + +## w3ds://auth (authentication) + +Signature-based, passwordless login. The user's eID Wallet signs a session ID the platform generated. The platform verifies the signature via the Registry + eVault. + +### URI format + +```text +w3ds://auth?redirect={callback}&session={sessionId}&platform={platformName} +``` + +- `redirect` — URL-encoded callback endpoint where the wallet POSTs the signed result. +- `session` — cryptographically random session ID (128-bit; UUIDv4 in practice). +- `platform` — display name. + +### Platform endpoints + +**`GET /api/auth/offer`** — issue the URI: + +```typescript +getOffer = async (req: Request, res: Response) => { + const url = new URL("/api/auth", process.env.PUBLIC_BASE_URL).toString(); + const session = uuidv4(); + const offer = `w3ds://auth?redirect=${url}&session=${session}&platform=YOUR_PLATFORM`; + res.json({ uri: offer }); +}; +``` + +Store `session` for 5 minutes to validate one-time-use. Response shape used by different platforms varies: eCurrency-api returns `{ offer, sessionId }`; Blabsy returns `{ uri: offer }`. Match your platform's client. + +**`POST /api/auth`** — receive the signed session and verify: + +Request body: + +```json +{ + "w3id": "@user-a.w3id", + "session": "550e8400-e29b-41d4-a716-446655440000", + "signature": "xK3vJZQ2...==", + "appVersion": "0.4.0" +} +``` + +Verification uses the `signature-validator` package (workspace): + +```typescript +import { verifySignature } from "signature-validator"; + +const verificationResult = await verifySignature({ + eName: w3id, + signature: signature, + payload: session, + registryBaseUrl: process.env.PUBLIC_REGISTRY_URL, +}); + +if (!verificationResult.valid) { + return res.status(401).json({ error: "Invalid signature", message: verificationResult.error }); +} +``` + +On success: mint your platform's session (JWT or cookie). On failure: 401. + +### appVersion + +Temporary field (will be sunset). Present because some early wallets signed differently. If you enforce it, use semver-compare against a minimum (e.g. `"0.4.0"`) and return 400 if too old. Delete this check once rollout completes. + +### Security + +- Session IDs must be cryptographically random (128 bits). +- One-time use; reject duplicates. +- Expire in ≤ 5 minutes. +- Return generic errors; never leak "user not found" vs "signature invalid". + +Detail: `docs/docs/W3DS Protocol/Authentication.md`. + +## w3ds://sign (arbitrary signatures) + +Same session-signing shape as auth, but for signing documents, votes, references, approvals, etc. + +### URI format + +```text +w3ds://sign?session={sessionId}&data={base64Data}&redirect_uri={encodedCallback} +``` + +- `data` — base64-encoded JSON `{ message, sessionId, ...context }`. Displayed to the user by the wallet. +- The wallet still signs only the **session ID**, never the full data. The platform uses the sessionId to look up the stored context. + +### Platform flow + +1. Platform receives sign request from client, generates sessionId, builds `data`, base64-encodes it, stores session with 15-minute TTL. +2. Returns `{ sessionId, qrData, expiresAt }` to the client for QR rendering. +3. User scans; wallet decodes `data`, shows message, requests confirmation. +4. Wallet POSTs to `redirect_uri`: + +```json +{ + "sessionId": "550e8400-...", + "signature": "xK3vJZQ2...==", + "w3id": "@user-a.w3id", + "message": "550e8400-..." // same as sessionId — for verification +} +``` + +5. Platform validates, verifies signature with `verifySignature(...)` using `message` as the `payload`, then processes the action and marks the session `completed` (or `security_violation`). + +Detail: `docs/docs/W3DS Protocol/Signing.md`. + +## Awareness Protocol (webhooks) + +Prototype-level fanout from eVault-core to every registered platform after a write. Fire-and-forget. Source: `docs/docs/W3DS Protocol/Awareness-Protocol.md`. + +### When it fires + +- After `createMetaEnvelope` (legacy `storeMetaEnvelope`): **3-second delay**, then fanout. The delay gives eVault time to reliably identify the requesting platform (from the Bearer token's `platform` claim) so it can exclude that platform from the fanout list. Without the delay you get "webhook ping-pong." +- After `updateMetaEnvelope` (legacy `updateMetaEnvelopeById`): **immediate** fanout. + +### Delivery mechanics + +1. eVault `GET /platforms` on the Registry → list of platform base URLs. +2. Filter out the requesting platform (normalized URL compare). +3. `POST {platformUrl}/api/webhook` on each remaining platform in parallel. +4. 5-second timeout per call. +5. `Promise.allSettled` — one failure does not affect others. +6. No retries. Failures are logged but do not block the mutation. + +### Packet format + +```json +{ + "id": "a1b2c3d4-...", + "w3id": "@e4d909c2-...", + "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "data": { + "content": "Hello, world!", + "mediaUrls": [], + "authorId": "@e4d909c2-...", + "createdAt": "2025-01-24T10:00:00Z" + } +} +``` + +Every platform receives every packet (broadcast). It is the platform's responsibility to inspect `schemaId` and drop packets it doesn't consume. A future revision will support ontology subscriptions and by-reference delivery. + +### Platform contract + +Every platform participating in W3DS MUST implement `POST /api/webhook` and: + +1. Find the mapping whose `schemaId` matches the packet's `schemaId`. +2. `adapter.fromGlobal({ data: body.data, mapping })` → local-shaped data. +3. Look up existing local ID for `body.id`; if found, update; otherwise create and persist the `(globalId, localId)` mapping. +4. Return 200. + +**Idempotency required**: the same `body.id` may arrive more than once (network retries, misbehaving eVault). Never create a second local entity for the same global ID. + +Full webhook controller code in [platform.md § Webhook controller](platform.md#webhook-controller). + +### Limitations to know + +- No retries. No ordering. No at-least-once guarantee. +- Recipient set = whatever `GET /platforms` returns. That is a prototype shortcut. + +For production, use Awareness-as-a-Service. + +### Awareness-as-a-Service (AaaS) + +Production-grade replacement layer. Source: `docs/docs/Services/Awareness-as-a-Service.md`. Key differences vs raw Awareness Protocol: + +- `POST /ingest` accepts packets from eVault-core. +- `GET /api/packets` — poll query with filters (ontology, eVault, time). +- Dynamic webhook subscriptions filtered by ontology or eVault. +- Retry engine with exponential backoff (30s → 24h). +- Dead-letter queue for permanently-failed deliveries. +- W3DS-authenticated public portal. + +Undifferentiated fanout → targeted delivery; no history → queryable; ungoverned access → access-controlled. + +## Signature formats + +Source: `docs/docs/W3DS Protocol/Signature-Formats.md`. + +### The algorithm + +Always **ECDSA P-256 (secp256r1) with SHA-256**. Payload is UTF-8 encoded, SHA-256'd (32 bytes), signed → 64 bytes raw `(r || s)`, each half 32 bytes. + +### Encodings on the wire + +| Origin | Encoding | Prefix | +|---|---|---| +| **Software keys** | Base64 of raw 64 bytes | none | +| **Hardware keys** (Passkey / Secure Enclave / HSM) | Multibase base58btc of DER-encoded ECDSA-Sig | `z` | + +Auto-detection at verify time: + +- Starts with `z` → multibase base58btc → decode → may be DER (starts with `0x30`, SEQUENCE) → normalize to raw 64 bytes. +- Starts with `m` → multibase base64 (no padding). +- Starts with `f` → hex, lowercase. +- Else → standard base64 / base64url. + +The `signature-validator` package handles all of this — you don't need to detect manually. Detection matters when you're implementing verification yourself. + +### Public key formats + +Multibase-encoded. Same prefix system: + +- `z` — base58btc +- `m` — base64, no padding +- `f` — hex, lowercase + +Content after decoding is one of: + +- **SPKI DER** — DER-encoded SubjectPublicKeyInfo (most common — what `crypto.subtle.exportKey('spki', ...)` emits). +- **Raw uncompressed** — 65 bytes: `0x04` + 32-byte X + 32-byte Y. + +Both are accepted by the verifier. + +## Signature verification recipe + +Full verification is a 5-step process. Use the `signature-validator` package unless you have a strong reason to reimplement. + +1. **Resolve eVault URL**: `GET {registryBaseUrl}/resolve?w3id=@user.w3id` → `{ uri }`. +2. **Fetch key binding certificates**: `GET {evaultUri}/whois` with header `X-ENAME: @user.w3id` → `{ keyBindingCertificates: [, ...] }`. +3. **Fetch Registry JWKS**: `GET {registryBaseUrl}/.well-known/jwks.json`. +4. **For each certificate**: + - Parse the JWT header/payload/signature. + - Verify JWT with the JWKS key matching `kid`. + - Check `exp` is in the future. + - Extract `publicKey` from the JWT payload (multibase-encoded). + - Decode the multibase public key. +5. **Verify the ECDSA signature**: import the decoded key, hash the payload with SHA-256, verify with ECDSA P-256. Return success on the first cert that verifies. + +The multi-cert loop handles multi-device users (each device has its own key). Any cert succeeding is enough. + +Using the SDK: + +```typescript +import { verifySignature } from "signature-validator"; + +const result = await verifySignature({ + eName: "@user.w3id", + signature: "z3K7vJZQ2F3k5L8mN9pQrS7tUvW1xY3zA5bC7dE9fG1hIjKlMnOpQrStUvWxYz", + payload: "550e8400-e29b-41d4-a716-446655440000", + registryBaseUrl: "https://registry.w3ds.metastate.foundation", +}); + +// result: { valid: boolean, error?: string, publicKey?: string } +``` + +## File URIs (`w3ds://file`) + +Standard URI scheme for referencing blobs. Source: `docs/docs/W3DS Protocol/File-URIs.md`. + +### Format + +```text +w3ds://file?id=@/ +``` + +Example: `w3ds://file?id=@alice/envelope-abc123`. + +### Storage layer + +Files uploaded via the eVault `uploadFile` mutation: + +1. Blob streamed to S3-compatible object storage (DigitalOcean Spaces) as `public-read`. +2. A **File Meta Envelope** is recorded with ontology `w3ds-file-v1`. Payload: + +```json +{ "filename", "contentType", "size", "blobKey", "publicUrl", "uploadedAt" } +``` + +3. The `w3ds://file` URI is built from the owner ename and the File Meta Envelope ID. + +Max size 50 MB decoded. `uploadFile` API signature in [evault.md § File upload](evault.md#file-upload). + +### Dereferencing + +**HTTP (eVault-core):** + +```http +GET {evaultUrl}/files/:metaEnvelopeId +X-ENAME: @ +``` + +Returns HTTP 302 redirect to the file's public object-storage URL. Only `http(s)` scheme allowed in the target. 400 on missing X-ENAME, malformed ID, or unsafe scheme. 404 on missing envelope or missing `publicUrl`. + +**Programmatic (Web3 Adapter):** + +```ts +import { dereferenceFileUri } from "@web3-adapter/w3ds/resolver"; + +const file = await dereferenceFileUri( + "w3ds://file?id=@alice/abc123", + evaultClient, +); +// { uri, ename, metaEnvelopeId, publicUrl, filename, contentType, size } +``` + +`parseFileUri` / `dereferenceFileUri` throw `InvalidW3dsUriError` on malformed URIs, wrong scheme, missing `id`, missing `@`, or non-existent ename / non-file envelope. + +### `w3ds-file-v1` vs `File` ontology — do not confuse + +Two distinct schemas. Conflating them is a common source of bugs. + +| | `w3ds-file-v1` | `File` ontology | +|---|---|---| +| Identifier | `w3ds-file-v1` (string literal) | `a1b2c3d4-e5f6-7890-abcd-ef1234567890` (UUID) | +| Created by | `uploadFile` mutation | Platform apps (file-manager, esigner) via Web3 Adapter mapping | +| Layer | Storage / transport — describes a blob | Application domain — a file record in a platform DB | +| Payload keys | `filename`, `contentType`, `size`, `blobKey`, `publicUrl`, `uploadedAt` | `id`, `name`, `displayName`, `description`, `mimeType`, `size`, `md5Hash`, `data`, `url`, `ownerId`, `folderId`, `createdAt`, `updatedAt` | +| Addressed by | `w3ds://file?id=@ename/` | Normal MetaEnvelope, synced through mapping | +| Schema location | `evault-core/src/core/utils/w3ds-uri.ts` (`FILE_SCHEMA_ID`) | `services/ontology/schemas/file.json` | + +A user-facing file may involve both: a `File` ontology record whose `url` points at a blob uploaded via `uploadFile`. + +### Mapper integration + +The Web3 Adapter's `__file(...)` mapping directive automatically calls `uploadFile` on `toGlobal` (replaces a `data:` URI or file value with a `w3ds://file` URI) and `dereferenceFileUri` on `fromGlobal` (replaces the URI with the public URL). Detail in [platform.md § Mapping directives](platform.md#mapping-directives). + +## References in the docs + +- Authentication: `docs/docs/W3DS Protocol/Authentication.md` +- Signing: `docs/docs/W3DS Protocol/Signing.md` +- Signature formats: `docs/docs/W3DS Protocol/Signature-Formats.md` +- Awareness Protocol: `docs/docs/W3DS Protocol/Awareness-Protocol.md` +- File URIs: `docs/docs/W3DS Protocol/File-URIs.md` +- Awareness-as-a-Service: `docs/docs/Services/Awareness-as-a-Service.md` diff --git a/skills/w3ds/reference/registry.md b/skills/w3ds/reference/registry.md new file mode 100644 index 000000000..fc52f9629 --- /dev/null +++ b/skills/w3ds/reference/registry.md @@ -0,0 +1,160 @@ +# Registry + Ontology + +The Registry is the discovery layer: it resolves W3IDs to service URLs, publishes JWKS, provides signed entropy for provisioning, and (temporarily) issues key-binding certificates. The Ontology service is the schema registry. Source: `docs/docs/Infrastructure/Registry.md`, `Ontology.md`. + +## Registry + +Production URL: `https://registry.w3ds.metastate.foundation`. Local dev port: **4321**. + +### GET /resolve?w3id=@\ + +Resolve an eName to its service endpoint. + +Response (200): + +```json +{ + "ename": "@user.w3id", + "uri": "https://resolved-service.example.com", + "evault": "evault-identifier", + "originalUri": "https://...", + "resolved": false +} +``` + +- `uri` — the endpoint to use for GraphQL / whois calls. +- `evault` — the eVault instance identifier (matches the `evaultId` returned by `/whois`). +- `resolved` — whether the URI was runtime-resolved (e.g. via health check). + +Errors: + +- 400 — missing `w3id` query param. +- 404 — no vault entry for that W3ID. + +Callers: Web3 Adapter's `EVaultClient` (before every store/update), the signature validator (before verifying), platforms (whenever they need the eVault URL for a user). + +### GET /list + +Returns every registered vault entry. No auth. Used by eVault-core to build the platform fanout list during Awareness Protocol delivery. Response is an array of `{ ename, uri, evault, originalUri, resolved }`. + +Note: some docs / code references call this `GET /platforms` — same concept. + +### GET /entropy + +Returns a signed ES256 JWT with 20 alphanumeric chars of entropy, used for provisioning. + +Response: + +```json +{ "token": "eyJhbGciOiJFUzI1NiIs..." } +``` + +JWT payload: `{ entropy: "<20 chars>", iat, exp }`. Valid 1 hour. Verify against `/.well-known/jwks.json`. + +### GET /.well-known/jwks.json + +Standard JWK set. Contains an EC P-256, ES256, `use: "sig"` key. Used to verify: + +- `/entropy` JWTs +- Key binding certificate JWTs served by eVault `/whois` + +### Key binding certificates (temporary — moving to Remote CA) + +The Registry currently issues JWTs binding an eName to a public key. Payload: `{ ename, publicKey, iat, exp }` (~1 hour TTL). Header: `{ alg: "ES256", kid: "entropy-key-1" }`. + +Flow: eVault stores a user's public key at provisioning time and internally requests a certificate from the Registry. The certificate is later served in `/whois` responses so verifiers can trust the eName↔publicKey binding without trusting the eVault directly. + +**Roadmap**: this responsibility moves to a Remote CA / Remote Notary. Treat the current Registry role as a prototype shortcut. + +## Ontology service + +Production URL: `https://ontology.w3ds.metastate.foundation`. + +### GET /schemas + +Returns a list of every registered schema: + +```json +[ + { "id": "550e8400-e29b-41d4-a716-446655440000", "title": "User" }, + { "id": "550e8400-e29b-41d4-a716-446655440001", "title": "SocialMediaPost" } +] +``` + +### GET /schemas/:id + +Returns the full JSON Schema (draft-07) for a schema W3ID. 404 if not found. + +Every schema must include: `schemaId` (W3ID), `title`, `type` (usually `"object"`), `properties`, `required`, `additionalProperties: false` (usually). + +Example: + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "schemaId": "550e8400-e29b-41d4-a716-446655440001", + "title": "SocialMediaPost", + "type": "object", + "properties": { + "id": { "type": "string", "format": "uri", "description": "W3ID" }, + "authorId": { "type": "string", "format": "uri", "description": "W3ID" }, + "content": { "type": "string" }, + "createdAt": { "type": "string", "format": "date-time" } + }, + "required": ["id", "authorId", "createdAt"], + "additionalProperties": false +} +``` + +In eVault, a `SocialMediaPost` MetaEnvelope has `ontology: "550e8400-e29b-41d4-a716-446655440001"`; its Envelopes have `fieldKey` values matching the schema's property names (`content`, `authorId`, `createdAt`, ...). + +### Human viewer + +- `GET /` — browser viewer with search (`?q=`). +- `GET /schema/:id` — permalink to one schema in the viewer. + +Use `/schemas` and `/schemas/:id` for programmatic access. + +## Canonical ontology W3IDs + +Memorize this table. These IDs appear in mapping.json files, in Awareness Protocol packets (`schemaId`), and in every eVault call — guessing them is guaranteed to be wrong. + +| Ontology | ID | +|---|---| +| **User** | `550e8400-e29b-41d4-a716-446655440000` | +| **SocialMediaPost** | `550e8400-e29b-41d4-a716-446655440001` | +| **Group** | `550e8400-e29b-41d4-a716-446655440003` | +| **Ledger** (eCurrency) | `550e8400-e29b-41d4-a716-446655440006` | +| **Currency** (eCurrency) | `550e8400-e29b-41d4-a716-446655440008` | +| **Account** (eCurrency) | `6fda64db-fd14-4fa2-bd38-77d2e5e6136d` | +| **Binding Document** | `b1d0a8c3-4e5f-6789-0abc-def012345678` | +| **File** (application layer) | `a1b2c3d4-e5f6-7890-abcd-ef1234567890` | +| **`w3ds-file-v1`** (storage layer) | `w3ds-file-v1` — **string literal, not a UUID** | + +Never confuse the `File` ontology and `w3ds-file-v1`. Different layers, different field names. Detail in [protocols.md § File URIs](protocols.md#file-uris-w3dsfile). + +If a user asks about an ontology not on this list, call `GET https://ontology.w3ds.metastate.foundation/schemas` to enumerate — do not guess. + +## Provisioner (adjacent, not part of Registry) + +Production URL: `https://provisioner.w3ds.metastate.foundation`. Local dev port: **3001** (co-hosted by eVault-core). + +`POST /provision` creates a new eVault. Detail in [wallet.md](wallet.md#provisioning). Body: + +```json +{ + "registryEntropy": "", + "namespace": "", + "verificationId": "", + "publicKey": "z..." // optional — omit for keyless (platform / group) eVaults +} +``` + +Response: `{ w3id, uri }`. + +## References in the docs + +- Registry endpoints + JWKS: `docs/docs/Infrastructure/Registry.md` +- Ontology API + schema format: `docs/docs/Infrastructure/Ontology.md` +- Provisioning flow: `docs/docs/Infrastructure/eID-Wallet.md`, `docs/docs/Infrastructure/wallet-sdk.md` +- Production URLs: `docs/docs/W3DS Basics/Links.md` diff --git a/skills/w3ds/reference/wallet.md b/skills/w3ds/reference/wallet.md new file mode 100644 index 000000000..a8fff8b45 --- /dev/null +++ b/skills/w3ds/reference/wallet.md @@ -0,0 +1,285 @@ +# eID Wallet + wallet-sdk + key delegation + +Everything about identity provisioning, signature creation, and key management across devices. Source: `docs/docs/Infrastructure/eID-Wallet.md`, `wallet-sdk.md`, `eVault-Key-Delegation.md`. + +## eID Wallet — what it is + +A **Tauri-based mobile app** (Rust + SvelteKit + TypeScript) that: + +- Generates and stores ECDSA P-256 key pairs. +- Uses hardware-backed key storage: **Secure Enclave** (iOS via LocalAuthentication), **HSM** (Android via KeyStore). Falls back to Web Crypto API for software keys. +- Provides the UI for onboarding, provisioning an eVault, and signing session IDs during platform auth and `w3ds://sign` flows. +- Uses `wallet-sdk` internally for the high-level flows. + +### Key manager selection + +The wallet picks its key manager per context: + +| Context | Manager | +|---|---| +| Pre-verification / test users | **Software** (Web Crypto API). | +| Real KYC-verified users | **Hardware only.** No software fallback — onboarding is blocked if hardware unavailable. | +| Non-onboarding operations | Explicit choice via configuration. | + +Contexts used by wallet code include `"onboarding"`, `"pre-verification"`, `"signing"`. + +### Signature output formats + +Determined by manager: + +- **Software** — base64-encoded raw 64-byte signature (r || s). +- **Hardware** — multibase base58btc (`z...`), often DER internally. + +Verifier auto-detects. Detail in [protocols.md § Signature formats](protocols.md#signature-formats). + +## Provisioning — onboarding a new eVault + +Endpoint: `POST /provision` on the Provisioner service. Local dev: port **3001** (co-hosted by eVault-core). Prod: `https://provisioner.w3ds.metastate.foundation`. + +### Request + +```http +POST /provision +Content-Type: application/json + +{ + "registryEntropy": "", + "namespace": "", + "verificationId": "", + "publicKey": "z..." // OPTIONAL — multibase, ECDSA P-256, SPKI DER inside +} +``` + +### Response + +```json +{ "w3id": "@e4d909c2-...", "uri": "https://evault.example.com/users/..." } +``` + +### `publicKey` — when to include + +- **User eVaults** — include it. The Provisioner passes it to eVault-core which stores the key and requests a Registry-issued key-binding certificate. Without a key, the eVault can't verify signatures on behalf of that user. +- **Keyless eVaults** (platforms, groups) — omit it. Some services don't need signature verification. + +### Full flow + +```text +1. keyService.generate("default", "onboarding") → hardware key pair +2. GET {registryUrl}/entropy → { token: JWT } +3. POST {provisionerUrl}/provision → { w3id, uri } + body: { registryEntropy, namespace, verificationId, publicKey } + (eVault-core stores publicKey and requests a key-binding cert from Registry) +4. Local wallet stores { w3id, uri, publicKey, privateKey } for later use +``` + +## wallet-sdk + +Location: `packages/wallet-sdk/`. Purpose: implement the high-level flows (provision, auth, key sync) while remaining crypto-agnostic via the `CryptoAdapter` interface. + +Dependencies: `jose` (JWT verification during sync). Uses global `fetch`. + +### `CryptoAdapter` — BYOC + +```typescript +interface CryptoAdapter { + getPublicKey(keyId: string, context: string): Promise; + signPayload(keyId: string, context: string, payload: string): Promise; + ensureKey(keyId: string, context: string): Promise<{ created: boolean }>; +} +``` + +- `getPublicKey` — multibase-encoded key or `undefined` if none. +- `signPayload` — signs and returns whatever encoding the adapter chooses (base64 for software, multibase for hardware). +- `ensureKey` — generates if missing; returns whether a new key was created. + +The eID Wallet wraps its KeyService in this interface via `createKeyServiceCryptoAdapter(keyService)` and exposes it on `GlobalState.walletSdkAdapter`. + +Contexts are opaque to the SDK — it just passes them through. + +### `provision(adapter, options)` + +Full end-to-end eVault provisioning. + +Steps: + +1. `GET {registryUrl}/entropy` → entropy token. +2. `adapter.getPublicKey(keyId, context)` → public key (must exist; call `ensureKey` first if needed). +3. `POST {provisionerUrl}/provision` with `{ registryEntropy, namespace, verificationId, publicKey }`. + +Options: `registryUrl`, `provisionerUrl`, `namespace`, `verificationId`, and optionally `keyId` (default `"default"`), `context` (default derived from `isPreVerification`), `isPreVerification`. + +Returns: `{ success, w3id, uri }`. Throws on HTTP or validation errors. + +```typescript +const result = await provision(globalState.walletSdkAdapter, { + registryUrl: PUBLIC_REGISTRY_URL, + provisionerUrl: PUBLIC_PROVISIONER_URL, + namespace: uuidv4(), + verificationId, + keyId: "default", + context: "pre-verification", + isPreVerification: true, +}); +// result.w3id, result.uri +``` + +### `authenticate(adapter, options)` + +Ensures the key exists and signs a session ID. The **caller** is responsible for POSTing the signature to the platform's `redirect` URL — the SDK does not do this for you. + +Steps: + +1. `adapter.ensureKey(keyId, context)`. +2. `adapter.signPayload(keyId, context, sessionId)`. +3. Return `{ signature }`. + +Options: `sessionId`, `context`, and optionally `keyId` (default `"default"`). + +```typescript +const { signature } = await authenticate(globalState.walletSdkAdapter, { + sessionId: sessionPayload, + keyId: "default", + context: isFake ? "pre-verification" : "onboarding", +}); + +// Then POST to the redirect URL from the w3ds://auth URI: +// { ename, session, signature, appVersion } +``` + +### `syncPublicKeyToEvault(adapter, options)` + +Syncs the adapter's public key to the eVault. Optionally skips PATCH if the current key is already present in a valid key-binding certificate (avoids unnecessary Registry round-trips). + +Steps: + +1. `GET {evaultUri}/whois` with `X-ENAME: {eName}` → `{ keyBindingCertificates: [...] }`. +2. If `registryUrl` is provided, verify each cert JWT with the Registry JWKS. If the current public key is already in a valid cert for this eName, skip the PATCH. +3. `adapter.getPublicKey(keyId, context)`. +4. `PATCH {evaultUri}/public-key` with `{ publicKey }` and headers `X-ENAME` (required), `Authorization: Bearer {authToken}` (optional). + +Options: `evaultUri`, `eName`, `context`, optionally `keyId` (default `"default"`), `authToken`, `registryUrl` (enables skip-if-present verification). + +The SDK does not read or write `localStorage`. Callers can persist a hint like `publicKeySaved_${eName}` themselves. + +```typescript +await syncPublicKeyToEvault(globalState.walletSdkAdapter, { + evaultUri: vault.uri, + eName, + keyId: "default", + context: isFake ? "pre-verification" : "onboarding", + authToken: PUBLIC_EID_WALLET_TOKEN || null, + registryUrl: PUBLIC_REGISTRY_URL, +}); +``` + +## Key delegation across devices + +W3IDs are loosely bound to keys — the W3ID never changes when keys do. This is what enables multi-device support, key rotation, and friend-based recovery. + +### Multi-device + +- Each device generates its own ECDSA P-256 key pair. +- Each device syncs its own public key to the same eVault (`PATCH /public-key`). +- eVault stores all keys and requests a key-binding certificate per key. +- Verifier iterates certificates until one matches — any device's signature verifies. + +### Key rotation + +Roadmap feature — not fully implemented today. + +Conceptual flow: + +1. Generate a new key on the new device. +2. `PATCH /public-key` to add it. +3. eVault requests a new key-binding certificate from the Registry. +4. Optionally revoke old keys (revocation UX is not built yet). + +The W3ID does not change during rotation. + +### Friend-based recovery + +A trust list (2–3 friends or notaries) can vouch for identity and approve key changes. The user defines the list while they still hold their keys. Not yet implemented — described in `docs/docs/W3DS Basics/W3ID.md`. + +## eVault endpoints used by the wallet + +### `PATCH /public-key` + +```http +PATCH /public-key +X-ENAME: @user.w3id +Authorization: Bearer +Content-Type: application/json + +{ "publicKey": "z3059301306072a8648ce3d020106082a8648ce3d03010703420004..." } +``` + +- 200 — key stored, key-binding certificate requested from Registry. +- 400 — missing X-ENAME or invalid body. +- 401 — invalid auth token. + +### `GET /whois` + +Returns `{ w3id, evaultId, keyBindingCertificates: [, ...] }`. Detail in [evault.md § GET /whois](evault.md#get-whois). + +## Desktop dev keys (no mobile wallet) + +For local development, you can generate ECDSA P-256 keys with the Web Crypto API, provision an eVault, and sign requests without touching a mobile device. The Dev Sandbox (`http://localhost:8080` after `pnpm dev:core`) implements this exact flow — see [dev-setup.md](dev-setup.md). + +### Manual outline + +```typescript +// 1. Generate keys +const keyPair = await crypto.subtle.generateKey( + { name: "ECDSA", namedCurve: "P-256" }, + true, + ["sign", "verify"], +); +const spki = await crypto.subtle.exportKey("spki", keyPair.publicKey); +const publicKey = "m" + btoa(String.fromCharCode(...new Uint8Array(spki))); // multibase base64 + +// 2. Provision +const { token: entropyToken } = await fetch(`${registryUrl}/entropy`).then(r => r.json()); +const { w3id, uri } = await fetch(`${provisionerUrl}/provision`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + registryEntropy: entropyToken, + namespace: crypto.randomUUID(), + verificationId: DEMO_VERIFICATION_ID, + publicKey, + }), +}).then(r => r.json()); + +// 3. Sign a session ID +const hash = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(sessionId)); +const sigBuf = await crypto.subtle.sign( + { name: "ECDSA", hash: "SHA-256" }, + keyPair.privateKey, + hash, +); +const signature = btoa(String.fromCharCode(...new Uint8Array(sigBuf))); // base64 +``` + +### Key storage (desktop) + +Store as JSON with `chmod 600`: + +```json +{ + "ename": "@e4d909c2-...", + "evaultUri": "https://evault.example.com/users/...", + "publicKey": "m...", + "privateKey": "", + "createdAt": "2025-01-24T10:00:00Z" +} +``` + +Never commit. Never reuse across environments. Never use desktop keys in production. + +## References in the docs + +- eID Wallet architecture: `docs/docs/Infrastructure/eID-Wallet.md` +- wallet-sdk API: `docs/docs/Infrastructure/wallet-sdk.md` +- Key delegation + `PATCH /public-key`: `docs/docs/Infrastructure/eVault-Key-Delegation.md` +- Desktop signing detail: `docs/docs/W3DS Protocol/Signature-Formats.md` +- Dev Sandbox (in-browser wallet substitute): `docs/docs/Post Platform Guide/dev-sandbox.md` From 2617582162d6e6efafc4d593621fee563d6ed18b Mon Sep 17 00:00:00 2001 From: coodos Date: Tue, 7 Jul 2026 20:14:42 +0530 Subject: [PATCH 2/4] docs: AI installation guide --- .../Post Platform Guide/ai-agent-skill.md | 211 ++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 docs/docs/Post Platform Guide/ai-agent-skill.md diff --git a/docs/docs/Post Platform Guide/ai-agent-skill.md b/docs/docs/Post Platform Guide/ai-agent-skill.md new file mode 100644 index 000000000..3710259b7 --- /dev/null +++ b/docs/docs/Post Platform Guide/ai-agent-skill.md @@ -0,0 +1,211 @@ +--- +sidebar_position: 7 +--- + +# AI Agent Skill (Claude Code, Codex, Cursor, Copilot, Windsurf, Aider) + +This repo ships a packaged **W3DS knowledge skill** under `skills/w3ds/` that you can load into your AI coding assistant so it stops guessing ontology UUIDs, mapping directives, and GraphQL field names. It's grounded in the docs you're reading now. + +Every agent tool has its own convention for repo-level context (skills, rules, `AGENTS.md`, `.cursorrules`, etc.). Install instructions below cover the ones people actually use. Pick your tool — or copy the fallback pattern for anything else. + +## What's in the skill + +- `SKILL.md` — router and ecosystem map +- `reference/evault.md` — GraphQL API, ACLs, `/whois`, `/logs` +- `reference/identity.md` — W3ID, eName, Binding Documents +- `reference/registry.md` — Registry endpoints, canonical ontology UUIDs +- `reference/protocols.md` — `w3ds://auth`, `w3ds://sign`, Awareness Protocol, signature formats, `w3ds://file` +- `reference/platform.md` — building a post-platform (auth, webhook, mapping directives, Web3 Adapter) +- `reference/wallet.md` — eID Wallet, wallet-sdk, key delegation +- `reference/dev-setup.md` — `pnpm dev:core` + debugging playbook + +## Claude Code + +### Option A — skills CLI (recommended) + +```bash +npx skills add MetaState-Prototype-Project/prototype@w3ds -g -y +``` + +`-g` installs globally to `~/.claude/skills/`. Drop `-g` for a project-local install. + +### Option B — symlink from a local clone + +If you already have the metastate repo checked out: + +```bash +ln -s "$(pwd)/skills/w3ds" ~/.claude/skills/w3ds +``` + +Edits under `skills/w3ds/` take effect on the next skill invocation — no re-symlink. + +### Option C — project-scoped `CLAUDE.md` + +Add a line to your project's `CLAUDE.md`: + +```markdown +When working on W3DS code, load `skills/w3ds/SKILL.md` from the metastate repo (or the installed skill) before answering. +``` + +Restart Claude Code after any install method. Verify with a question like *"how do I write a webhook controller for a W3DS post-platform?"* — the skill should be picked up. + +## OpenAI Codex CLI (`AGENTS.md`) + +Codex CLI reads `AGENTS.md` from the repo root and `~/.codex/AGENTS.md` for user-level context. + +### Project-scoped + +Copy the skill content into `AGENTS.md` at the root of the project you're building on W3DS: + +```bash +cat skills/w3ds/SKILL.md > AGENTS.md +echo -e "\n\n---\n" >> AGENTS.md +for f in skills/w3ds/reference/*.md; do + echo -e "\n## $(basename "$f" .md)\n" >> AGENTS.md + cat "$f" >> AGENTS.md +done +``` + +Or, if `AGENTS.md` already exists, append the skill as a section: + +```bash +echo -e "\n\n# W3DS reference\n" >> AGENTS.md +cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md >> AGENTS.md +``` + +### User-scoped + +Put the same concatenated content in `~/.codex/AGENTS.md` if you want it available in every project you touch. + +## Cursor (`.cursor/rules/`) + +Cursor uses `.cursor/rules/*.mdc` files. Each rule file has YAML frontmatter controlling when it activates. + +Create `.cursor/rules/w3ds.mdc`: + +```mdc +--- +description: W3DS (Web 3 Data Spaces) knowledge — eVault GraphQL, Web3 Adapter, w3ds://auth, w3ds://sign, mapping directives, ontology UUIDs +globs: + - "**/*.ts" + - "**/*.tsx" + - "**/mapping*.json" + - "**/AGENTS.md" +alwaysApply: false +--- + + + +--- + + +``` + +Or generate it with: + +```bash +mkdir -p .cursor/rules +{ + echo '---' + echo 'description: W3DS (Web 3 Data Spaces) knowledge — eVault GraphQL, Web3 Adapter, w3ds://auth, w3ds://sign, mapping directives, ontology UUIDs' + echo 'globs:' + echo ' - "**/*.ts"' + echo ' - "**/*.tsx"' + echo ' - "**/mapping*.json"' + echo 'alwaysApply: false' + echo '---' + echo + tail -n +6 skills/w3ds/SKILL.md + echo + for f in skills/w3ds/reference/*.md; do + echo -e "\n---\n\n# $(basename "$f" .md)\n" + cat "$f" + done +} > .cursor/rules/w3ds.mdc +``` + +Set `alwaysApply: true` if you want the rule loaded for every request instead of matching on globs. + +## GitHub Copilot (`.github/copilot-instructions.md`) + +Copilot reads `.github/copilot-instructions.md` for repo-level guidance. + +```bash +mkdir -p .github +cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .github/copilot-instructions.md +``` + +Commit the file. Copilot picks it up automatically for repositories that have it enabled in settings (Copilot → Chat → *Instructions*). + +## Windsurf (`.windsurfrules`) + +Windsurf reads `.windsurfrules` at the repo root: + +```bash +cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .windsurfrules +``` + +For user-level rules, put the same content in `~/.codeium/windsurf/memories/global_rules.md`. + +## Aider (`CONVENTIONS.md` + `--read`) + +Aider doesn't auto-load a file, but you can pin it: + +```bash +aider --read skills/w3ds/SKILL.md \ + --read skills/w3ds/reference/platform.md \ + --read skills/w3ds/reference/evault.md +``` + +For long-running sessions, drop everything into `CONVENTIONS.md` and start Aider with: + +```bash +aider --read CONVENTIONS.md +``` + +## Continue.dev, Cline, Roo, and others + +These agents accept some form of system-prompt or context-injection. The simplest universal pattern: + +1. Concatenate the skill into one markdown file: + + ```bash + cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > w3ds-context.md + ``` + +2. Add `w3ds-context.md` to whatever the agent uses for repo-level context: + - **Continue.dev** — reference it in `.continue/context/` or attach with `@Files`. + - **Cline** — put in `.clinerules` or `.clinerules-*`. + - **Roo** — same as Cline (`.clinerules`). + - **Anything else** — most agents accept a system prompt or a "read this file" flag. Point at `w3ds-context.md`. + +## Any tool — the pattern + +If your tool isn't listed above, the pattern is always the same: + +1. Concatenate `skills/w3ds/SKILL.md` and `skills/w3ds/reference/*.md` into whatever file the tool reads for repo instructions. +2. If the tool supports rule-file frontmatter (Cursor, some others), keep it descriptive so the tool knows when to activate the rule. +3. If the tool has no rule system at all, point it at the skill in your prompt: *"Use `skills/w3ds/` in this repo as authoritative W3DS reference before answering."* + +## Updating + +The skill mirrors the docs. When docs change, pull the latest metastate `main` and: + +- **Claude Code (skills CLI):** `npx skills update` +- **Claude Code (symlink):** nothing — edits take effect immediately. +- **Copy-based installs (Cursor, Copilot, Windsurf, Codex, Aider):** re-run the concatenation command from the section above. + +If you're building on a fork, add a repo hook or a pre-commit step that re-runs the concatenation so the copy in your project stays fresh. + +## Contributing + +Gaps or wrong answers? PRs welcome. The skill lives at `skills/w3ds/` in this repo. Rules of thumb: + +- Ground every claim in a `docs/docs/...` path. +- Keep the main `SKILL.md` scannable (under ~200 lines); push detail into `reference/*.md`. +- Don't invent APIs. If the docs don't say it, don't put it in the skill. + +## Reference + +- Skill source: [`skills/w3ds/`](https://github.com/MetaState-Prototype-Project/prototype/tree/main/skills/w3ds) in the metastate repo. +- Distribution readme: [`skills/README.md`](https://github.com/MetaState-Prototype-Project/prototype/tree/main/skills). From 2f73b903dd03cca4ea8ce1c6ad689b29cc3048c7 Mon Sep 17 00:00:00 2001 From: coodos Date: Tue, 7 Jul 2026 20:17:49 +0530 Subject: [PATCH 3/4] docs: windows --- .../Post Platform Guide/ai-agent-skill.md | 121 ++++++++++++++++-- 1 file changed, 109 insertions(+), 12 deletions(-) diff --git a/docs/docs/Post Platform Guide/ai-agent-skill.md b/docs/docs/Post Platform Guide/ai-agent-skill.md index 3710259b7..8ccf0d0b2 100644 --- a/docs/docs/Post Platform Guide/ai-agent-skill.md +++ b/docs/docs/Post Platform Guide/ai-agent-skill.md @@ -2,12 +2,22 @@ sidebar_position: 7 --- -# AI Agent Skill (Claude Code, Codex, Cursor, Copilot, Windsurf, Aider) +# AI Agent Skill This repo ships a packaged **W3DS knowledge skill** under `skills/w3ds/` that you can load into your AI coding assistant so it stops guessing ontology UUIDs, mapping directives, and GraphQL field names. It's grounded in the docs you're reading now. Every agent tool has its own convention for repo-level context (skills, rules, `AGENTS.md`, `.cursorrules`, etc.). Install instructions below cover the ones people actually use. Pick your tool — or copy the fallback pattern for anything else. +:::note Windows users + +Command blocks are labeled **macOS / Linux (bash)** and **Windows (PowerShell)** where they differ. If you use **WSL** or **Git Bash**, the bash commands work verbatim — skip the PowerShell variants. + +- Paths written `~/.foo/bar` also work in PowerShell (`~` resolves to `$HOME` = `%USERPROFILE%`). +- Symlinks on Windows require either an **Administrator** PowerShell session **or** [Developer Mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging) enabled in Settings. +- Forward slashes in paths are accepted by `npx`, `node`, `aider`, and most cross-platform CLIs on Windows — only PowerShell-native cmdlets prefer backslashes. + +::: + ## What's in the skill - `SKILL.md` — router and ecosystem map @@ -33,10 +43,20 @@ npx skills add MetaState-Prototype-Project/prototype@w3ds -g -y If you already have the metastate repo checked out: +**macOS / Linux (bash):** + ```bash ln -s "$(pwd)/skills/w3ds" ~/.claude/skills/w3ds ``` +**Windows (PowerShell, Administrator or Developer Mode):** + +```powershell +New-Item -ItemType SymbolicLink ` + -Path "$HOME\.claude\skills\w3ds" ` + -Target "$PWD\skills\w3ds" +``` + Edits under `skills/w3ds/` take effect on the next skill invocation — no re-symlink. ### Option C — project-scoped `CLAUDE.md` @@ -49,13 +69,15 @@ When working on W3DS code, load `skills/w3ds/SKILL.md` from the metastate repo ( Restart Claude Code after any install method. Verify with a question like *"how do I write a webhook controller for a W3DS post-platform?"* — the skill should be picked up. -## OpenAI Codex CLI (`AGENTS.md`) +## OpenAI Codex CLI Codex CLI reads `AGENTS.md` from the repo root and `~/.codex/AGENTS.md` for user-level context. ### Project-scoped -Copy the skill content into `AGENTS.md` at the root of the project you're building on W3DS: +Copy the skill content into `AGENTS.md` at the root of the project you're building on W3DS. + +**macOS / Linux (bash):** ```bash cat skills/w3ds/SKILL.md > AGENTS.md @@ -66,18 +88,38 @@ for f in skills/w3ds/reference/*.md; do done ``` -Or, if `AGENTS.md` already exists, append the skill as a section: +**Windows (PowerShell):** + +```powershell +Get-Content skills/w3ds/SKILL.md | Set-Content AGENTS.md +Add-Content AGENTS.md "`n`n---`n" +Get-ChildItem skills/w3ds/reference/*.md | ForEach-Object { + Add-Content AGENTS.md "`n## $($_.BaseName)`n" + Get-Content $_.FullName | Add-Content AGENTS.md +} +``` + +Or, if `AGENTS.md` already exists, append the skill as a section. + +**macOS / Linux (bash):** ```bash echo -e "\n\n# W3DS reference\n" >> AGENTS.md cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md >> AGENTS.md ``` +**Windows (PowerShell):** + +```powershell +Add-Content AGENTS.md "`n`n# W3DS reference`n" +Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Add-Content AGENTS.md +``` + ### User-scoped Put the same concatenated content in `~/.codex/AGENTS.md` if you want it available in every project you touch. -## Cursor (`.cursor/rules/`) +## Cursor Cursor uses `.cursor/rules/*.mdc` files. Each rule file has YAML frontmatter controlling when it activates. @@ -101,7 +143,9 @@ alwaysApply: false ``` -Or generate it with: +Or generate it. + +**macOS / Linux (bash):** ```bash mkdir -p .cursor/rules @@ -124,30 +168,75 @@ mkdir -p .cursor/rules } > .cursor/rules/w3ds.mdc ``` +**Windows (PowerShell):** + +```powershell +New-Item -ItemType Directory -Force -Path .cursor/rules | Out-Null +$out = '.cursor/rules/w3ds.mdc' + +@' +--- +description: W3DS (Web 3 Data Spaces) knowledge — eVault GraphQL, Web3 Adapter, w3ds://auth, w3ds://sign, mapping directives, ontology UUIDs +globs: + - "**/*.ts" + - "**/*.tsx" + - "**/mapping*.json" +alwaysApply: false +--- + +'@ | Set-Content $out + +Get-Content skills/w3ds/SKILL.md | Select-Object -Skip 5 | Add-Content $out +Get-ChildItem skills/w3ds/reference/*.md | ForEach-Object { + Add-Content $out "`n---`n`n# $($_.BaseName)`n" + Get-Content $_.FullName | Add-Content $out +} +``` + Set `alwaysApply: true` if you want the rule loaded for every request instead of matching on globs. -## GitHub Copilot (`.github/copilot-instructions.md`) +## GitHub Copilot Copilot reads `.github/copilot-instructions.md` for repo-level guidance. +**macOS / Linux (bash):** + ```bash mkdir -p .github cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .github/copilot-instructions.md ``` +**Windows (PowerShell):** + +```powershell +New-Item -ItemType Directory -Force -Path .github | Out-Null +Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | + Set-Content .github/copilot-instructions.md +``` + Commit the file. Copilot picks it up automatically for repositories that have it enabled in settings (Copilot → Chat → *Instructions*). -## Windsurf (`.windsurfrules`) +## Windsurf + +Windsurf reads `.windsurfrules` at the repo root. -Windsurf reads `.windsurfrules` at the repo root: +**macOS / Linux (bash):** ```bash cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > .windsurfrules ``` -For user-level rules, put the same content in `~/.codeium/windsurf/memories/global_rules.md`. +**Windows (PowerShell):** + +```powershell +Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Set-Content .windsurfrules +``` + +For user-level rules, put the same content in: +- macOS / Linux: `~/.codeium/windsurf/memories/global_rules.md` +- Windows: `$HOME\.codeium\windsurf\memories\global_rules.md` -## Aider (`CONVENTIONS.md` + `--read`) +## Aider Aider doesn't auto-load a file, but you can pin it: @@ -167,12 +256,20 @@ aider --read CONVENTIONS.md These agents accept some form of system-prompt or context-injection. The simplest universal pattern: -1. Concatenate the skill into one markdown file: +1. Concatenate the skill into one markdown file. + + **macOS / Linux (bash):** ```bash cat skills/w3ds/SKILL.md skills/w3ds/reference/*.md > w3ds-context.md ``` + **Windows (PowerShell):** + + ```powershell + Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Set-Content w3ds-context.md + ``` + 2. Add `w3ds-context.md` to whatever the agent uses for repo-level context: - **Continue.dev** — reference it in `.continue/context/` or attach with `@Files`. - **Cline** — put in `.clinerules` or `.clinerules-*`. From 5e15b507e70542e681a92c396ae3337e8a4fdb76 Mon Sep 17 00:00:00 2001 From: coodos Date: Tue, 7 Jul 2026 20:22:26 +0530 Subject: [PATCH 4/4] docs: skills.sh --- .../Post Platform Guide/ai-agent-skill.md | 95 +++++++++++++++---- skills/README.md | 54 +++++++++-- 2 files changed, 126 insertions(+), 23 deletions(-) diff --git a/docs/docs/Post Platform Guide/ai-agent-skill.md b/docs/docs/Post Platform Guide/ai-agent-skill.md index 8ccf0d0b2..74e953f31 100644 --- a/docs/docs/Post Platform Guide/ai-agent-skill.md +++ b/docs/docs/Post Platform Guide/ai-agent-skill.md @@ -6,7 +6,7 @@ sidebar_position: 7 This repo ships a packaged **W3DS knowledge skill** under `skills/w3ds/` that you can load into your AI coding assistant so it stops guessing ontology UUIDs, mapping directives, and GraphQL field names. It's grounded in the docs you're reading now. -Every agent tool has its own convention for repo-level context (skills, rules, `AGENTS.md`, `.cursorrules`, etc.). Install instructions below cover the ones people actually use. Pick your tool — or copy the fallback pattern for anything else. +The easiest install for every supported agent is the [`npx skills`](https://skills.sh) CLI — it targets Claude Code, Codex, Cursor, GitHub Copilot, Windsurf, OpenCode, Cline, Gemini, and 60+ others. Manual per-tool instructions are further down if you'd rather bypass the CLI or your agent isn't supported yet. :::note Windows users @@ -29,17 +29,72 @@ Command blocks are labeled **macOS / Linux (bash)** and **Windows (PowerShell)** - `reference/wallet.md` — eID Wallet, wallet-sdk, key delegation - `reference/dev-setup.md` — `pnpm dev:core` + debugging playbook -## Claude Code +## Install with `npx skills` (all tools) -### Option A — skills CLI (recommended) +The [skills CLI](https://skills.sh) auto-detects the AI coding agents you have installed and configures each of them. Works cross-platform (macOS / Linux / Windows PowerShell / WSL). + +### Recommended + +```bash +npx skills add MetaState-Prototype-Project/prototype@w3ds +``` + +The CLI detects your installed agents and prompts for which to target. Default install is **project-local** (committed with your project, shared with your team); pass `-g` for a global install. + +### Pick a specific tool + +Skip the prompt with `-a, --agent`: ```bash -npx skills add MetaState-Prototype-Project/prototype@w3ds -g -y +# Claude Code +npx skills add MetaState-Prototype-Project/prototype@w3ds -a claude-code + +# OpenAI Codex CLI +npx skills add MetaState-Prototype-Project/prototype@w3ds -a codex + +# Cursor +npx skills add MetaState-Prototype-Project/prototype@w3ds -a cursor + +# GitHub Copilot +npx skills add MetaState-Prototype-Project/prototype@w3ds -a copilot + +# Windsurf +npx skills add MetaState-Prototype-Project/prototype@w3ds -a windsurf + +# OpenCode +npx skills add MetaState-Prototype-Project/prototype@w3ds -a opencode + +# Every supported agent installed on your machine +npx skills add MetaState-Prototype-Project/prototype@w3ds --all ``` -`-g` installs globally to `~/.claude/skills/`. Drop `-g` for a project-local install. +Full agent list at [skills.sh](https://skills.sh) (Gemini, Cline, Roo, Zed, Goose, Kilo, VS Code, etc. are all supported). + +### Common flags -### Option B — symlink from a local clone +- `-g` — install globally to `~//skills/` (default: project-local `.//skills/`). +- `-a, --agent ` — target one or more specific agents (repeatable / space-separated). +- `--all` — install to every supported agent detected on your machine. +- `--copy` — copy files instead of symlinking. +- `-y, --yes` — skip confirmation prompts. + +### Use without installing + +Load the skill into a single session without touching your filesystem: + +```bash +# Pipe the generated prompt into your agent +npx skills use MetaState-Prototype-Project/prototype@w3ds | claude + +# Or start any supported agent interactively with the skill loaded +npx skills use MetaState-Prototype-Project/prototype@w3ds --agent cursor +``` + +## Claude Code (manual) + +If you'd rather not use the CLI, or you want to hack on the skill locally: + +### Option A — symlink from a local clone If you already have the metastate repo checked out: @@ -59,7 +114,7 @@ New-Item -ItemType SymbolicLink ` Edits under `skills/w3ds/` take effect on the next skill invocation — no re-symlink. -### Option C — project-scoped `CLAUDE.md` +### Option B — project-scoped `CLAUDE.md` Add a line to your project's `CLAUDE.md`: @@ -69,7 +124,9 @@ When working on W3DS code, load `skills/w3ds/SKILL.md` from the metastate repo ( Restart Claude Code after any install method. Verify with a question like *"how do I write a webhook controller for a W3DS post-platform?"* — the skill should be picked up. -## OpenAI Codex CLI +## OpenAI Codex CLI (manual) + +Simplest install is `npx skills add MetaState-Prototype-Project/prototype@w3ds -a codex` from the section above. Everything below is for when you want to author `AGENTS.md` by hand. Codex CLI reads `AGENTS.md` from the repo root and `~/.codex/AGENTS.md` for user-level context. @@ -119,7 +176,9 @@ Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Add-Content AGENT Put the same concatenated content in `~/.codex/AGENTS.md` if you want it available in every project you touch. -## Cursor +## Cursor (manual) + +Simplest install is `npx skills add MetaState-Prototype-Project/prototype@w3ds -a cursor` from the section above. Everything below is for when you want a hand-tuned `.mdc` file. Cursor uses `.cursor/rules/*.mdc` files. Each rule file has YAML frontmatter controlling when it activates. @@ -195,7 +254,9 @@ Get-ChildItem skills/w3ds/reference/*.md | ForEach-Object { Set `alwaysApply: true` if you want the rule loaded for every request instead of matching on globs. -## GitHub Copilot +## GitHub Copilot (manual) + +Simplest install is `npx skills add MetaState-Prototype-Project/prototype@w3ds -a copilot` from the section above. Everything below is for when you want to write `.github/copilot-instructions.md` yourself. Copilot reads `.github/copilot-instructions.md` for repo-level guidance. @@ -216,7 +277,9 @@ Get-Content skills/w3ds/SKILL.md, skills/w3ds/reference/*.md | Commit the file. Copilot picks it up automatically for repositories that have it enabled in settings (Copilot → Chat → *Instructions*). -## Windsurf +## Windsurf (manual) + +Simplest install is `npx skills add MetaState-Prototype-Project/prototype@w3ds -a windsurf` from the section above. Everything below is for when you want to write `.windsurfrules` yourself. Windsurf reads `.windsurfrules` at the repo root. @@ -254,7 +317,7 @@ aider --read CONVENTIONS.md ## Continue.dev, Cline, Roo, and others -These agents accept some form of system-prompt or context-injection. The simplest universal pattern: +Cline, Roo, Continue.dev, Gemini, Zed, Goose, Kilo, and dozens more are all supported by `npx skills` — try `-a ` from the [main install section](#install-with-npx-skills-all-tools) first. If your agent isn't supported yet or you want to bypass the CLI, use this universal pattern: 1. Concatenate the skill into one markdown file. @@ -288,11 +351,11 @@ If your tool isn't listed above, the pattern is always the same: The skill mirrors the docs. When docs change, pull the latest metastate `main` and: -- **Claude Code (skills CLI):** `npx skills update` -- **Claude Code (symlink):** nothing — edits take effect immediately. -- **Copy-based installs (Cursor, Copilot, Windsurf, Codex, Aider):** re-run the concatenation command from the section above. +- **`npx skills` install (any agent):** `npx skills update` — updates every installed skill across every agent. +- **Symlink install (Claude Code):** nothing — edits take effect immediately. +- **Manual copy install (Cursor, Copilot, Windsurf, Codex, Aider):** re-run the concatenation command from the relevant section above. -If you're building on a fork, add a repo hook or a pre-commit step that re-runs the concatenation so the copy in your project stays fresh. +If you're building on a fork and shipping the manual copy, add a repo hook or pre-commit step that re-runs the concatenation so the copy in your project stays fresh. ## Contributing diff --git a/skills/README.md b/skills/README.md index 33356f0f6..ef3fe263d 100644 --- a/skills/README.md +++ b/skills/README.md @@ -1,6 +1,6 @@ # MetaState Skills -Installable Claude Code skills for the MetaState / W3DS ecosystem. +Installable AI-agent skills for the MetaState / W3DS ecosystem. Powered by the [skills.sh](https://skills.sh) CLI — works with Claude Code, Codex, Cursor, GitHub Copilot, Windsurf, OpenCode, Cline, Gemini, and 60+ other coding agents. ## Available skills @@ -10,26 +10,66 @@ Installable Claude Code skills for the MetaState / W3DS ecosystem. ## Install -Skills in this directory follow the [skills.sh](https://skills.sh/) `/@` layout and are installable via the `skills` CLI: +Skills in this directory follow the [skills.sh](https://skills.sh/) `/@` layout. ```bash -npx skills add MetaState-Prototype-Project/prototype@w3ds -g -y +npx skills add MetaState-Prototype-Project/prototype@w3ds ``` -`-g` installs into `~/.claude/skills/`. Omit for a project-local install. +Auto-detects the agents you have installed and prompts for which to target. + +### Common flags + +- `-g` — install globally to `~//skills/` (default is project-local `.//skills/`). +- `-a, --agent ` — target a specific agent (`claude-code`, `codex`, `cursor`, `copilot`, `windsurf`, `opencode`, etc.). +- `--all` — install to every supported agent detected on your machine. +- `--copy` — copy files instead of symlinking. +- `-y, --yes` — skip confirmation prompts. + +### Examples + +```bash +# Install globally for Claude Code only +npx skills add MetaState-Prototype-Project/prototype@w3ds -g -a claude-code + +# Install for both Cursor and Codex, project-local +npx skills add MetaState-Prototype-Project/prototype@w3ds -a cursor -a codex + +# Install for every supported agent on the machine +npx skills add MetaState-Prototype-Project/prototype@w3ds --all -y +``` + +### Use without installing + +```bash +npx skills use MetaState-Prototype-Project/prototype@w3ds | claude +npx skills use MetaState-Prototype-Project/prototype@w3ds --agent cursor +``` + +Full per-tool install guide (manual paths for agents not yet covered by the CLI, or if you'd rather bypass it) lives at [docs/Post Platform Guide/AI Agent Skill](../docs/docs/Post%20Platform%20Guide/ai-agent-skill.md). ## Local development -To hack on a skill without publishing, symlink it into the Claude skills directory: +To hack on a skill without publishing, symlink it into your agent's skills directory. For Claude Code: + +**macOS / Linux:** ```bash ln -s "$(pwd)/skills/w3ds" ~/.claude/skills/w3ds ``` -Restart Claude Code so the new skill is picked up. Edits to files under `skills/w3ds/` take effect immediately on the next skill invocation — no re-symlink needed. +**Windows (PowerShell, Administrator or Developer Mode):** + +```powershell +New-Item -ItemType SymbolicLink ` + -Path "$HOME\.claude\skills\w3ds" ` + -Target "$PWD\skills\w3ds" +``` + +Edits to files under `skills/w3ds/` take effect on the next skill invocation — no re-symlink needed. Restart your agent so the new skill is picked up. ## Authoring notes -Each skill is a directory with a top-level `SKILL.md` and optional `reference/` files. The `SKILL.md` frontmatter needs at minimum a `name` and a `description`; the description is what Claude uses to decide when to trigger the skill, so it should list the concrete surfaces the skill covers (concepts, APIs, protocol names, common questions). +Each skill is a directory with a top-level `SKILL.md` and optional `reference/` files. The `SKILL.md` frontmatter needs at minimum a `name` and a `description`; the description is what the agent uses to decide when to trigger the skill, so list the concrete surfaces it covers (concepts, APIs, protocol names, common questions). Keep the main `SKILL.md` scannable (~150 lines) and push deep content into `reference/*.md` files that get loaded on demand.