Skip to content

History / Data Access Boundary

Revisions

  • wiki: refresh diagrams to reflect current implementation state After Phase 1.a + 1.b + 2.a.1 shipped, the diagrams update to show actual state vs the original "design-only" sketches: - Home — Today's topology diagram refreshed: - noetl-server box shows both Python + Rust versions with ✅ on /api/internal/* completion - System worker pool added to the deploy map with NOETL_INTERNAL_ API_TOKEN annotation - outbox-publisher + projector marked "↓ to-be-retired" - "What's live on kind right now" callout below the diagram - Umbrella-System-Pool-Design — two new diagrams + one refresh: - NEW "Implementation status (2026-06-02)" ASCII matrix showing Phase 1.a / 1.b / 2.a.1 ✅ and 2.a.2 / 2.a.3 / 2.b / 3 / 4 ⏳ - NEW "Auth flow — Secret to API call" diagram showing the five-hop pipeline from K8s Secret through env mount, keychain-env-allowlist, AuthResolver, HTTPS, server gate - Existing system playbook execution flow diagram unchanged (the contract didn't shift) - Data-Access-Boundary — adds the auth flow diagram (same five- hop pipeline) with explicit "configuration-only, no Rust code" callout below. This is the page operators land on when wondering "how does a system playbook get authenticated." The repeated auth-flow diagram is intentional — Data-Access- Boundary is the convention page (rule-of-thumb readers); Umbrella- System-Pool-Design is the umbrella page (people picking up Phase 2 work). Both audiences benefit from the same diagram in their landing context. Per agents/rules/wiki-maintenance.md Rule 0a: diagrams update in the same change set as the work that produces them. This commit catches up the diagrams to the actual state after Phase 2.a.1 shipped.

    @kadyapam kadyapam committed Jun 2, 2026
  • add architectural diagrams across dashboard pages After session 2026-06-02 (late afternoon) request: "add that diagrams too" — make the architecture readable at a glance from any wiki page a developer might land on. Four diagrams added: 1. **Two-umbrella interlock** (#49 + #46) — visual of how the Rust server port's Phase C unblocks the system pool's Phase 2. Same API surface produced by one, consumed by the other. Added to: - Home.md (architecture-at-a-glance section) - Umbrella-Rust-Server-Port.md - Umbrella-System-Pool-Design.md 2. **Long-term Python trajectory** — three-column view: today / after #46 Phase 2 / long-term. Shows Python becoming a container payload, not a platform runtime. Added to: - Home.md - Umbrella-System-Pool-Design.md 3. **Data access boundary** — workers + system worker pool call server HTTP API; only the server talks to Postgres directly; external-subsystem playbooks (auth) go direct to external systems. Added to: - Data-Access-Boundary.md (convention page) 4. **Today's deploy topology** (pre-existing, kept) — gateway → server → outbox/projector + NATS → worker pools. Already on Home.md. Together these give a developer landing on Home immediately: - What's deployed today - The two primary in-flight umbrellas + how they interlock - The Python footprint trajectory Each Umbrella-* page carries its relevant subset so a developer opening a specific umbrella also gets the visual context. Issue comments mirror these diagrams: - #49 — interlock diagram (gh issue comment 4605222980) - #46 — interlock + trajectory diagrams (gh issue comment 4605227009)

    @kadyapam kadyapam committed Jun 2, 2026
  • add Data-Access-Boundary convention page + corrected playbook examples After 2026-06-02 (afternoon) standing instruction: NoETL platform data is accessible via server API only. Workers (including system worker pool) call the API, never the DB directly. Changes: - Data-Access-Boundary.md — new convention page pointing to the authoritative rule in agents/rules/. TL;DR + why (connection pool, sharding, consistency) + auth exception + decision tree. - _Sidebar.md — adds Data Access Boundary under Conventions. - Home.md — adds Data Access Boundary to the conventions index with link reference. - Umbrella-System-Pool-Design.md: - Adds "Data access boundary" section explaining the rule. - Phase 1 split into 1.a (server-side API endpoints) and 1.b (system worker pool deployment). 1.a lists the five new /api/internal/* endpoints needed before any Phase 2 playbook can run. - Phase 2 playbook examples corrected: - system/outbox_publisher uses tool: http (claim/mark) + tool: nats (publish). - system/projector uses tool: nats (pull batch) + tool: http (project).

    @kadyapam kadyapam committed Jun 2, 2026