subscribe: add noetl subscribe local-mode listener page (RFC #90 Phase 6)

kadyapam committed Jun 12, 2026

wiki: new noetl-events page + executor + sidebar + home cross-links (EE-4) Catch-up wiki update for the EE-4 sequence that already landed on the code side (cli#49, cli#50, server#38). - New top-level page `noetl-events.md` documenting the workspace crate published as 0.1.0 on 2026-06-05: purpose, public API (`ExecutorEvent`, `EventSink`, `EventEmitter`, `NoopSink`), wire- format alignment with the Python `EventEmitRequest`, the re-export contract from `noetl-executor::events`, the path+version dep recipe, release history. - `_Sidebar.md` Architecture section now lists both crates. - `executor-crate-architecture.md` workspace table adds the noetl-events row; source-layout tree shows the `events/` folder + flags `executor/src/events.rs` as a 1-line re-export since 0.4.0. - `Home.md` Distribution row mentions noetl-events on crates.io; new Architecture section pointing at both crate pages. Tracks the EE-4 sequence under noetl/ai-meta#49 + noetl/cli#51 (R1 sub-issue) + noetl/cli#52 (R2 sub-issue).

kadyapam committed Jun 4, 2026

docs(executor-arch): add GCS sink object_store semantic-divergence rows (noetl/ai-meta#31) Adds two rows to the semantic-divergence table covering the migration of SinkTarget::Gcs from gsutil subprocess to object_store crate in noetl/cli#48: - GCS upload transport change (gsutil → GoogleCloudStorageBuilder) - GCS credential resolution change (gsutil gcloud store → from_env() ADC chain) Also marks the GCS sink → object_store deferred item as complete and updates the R-1.1 "next phases" note. Refs noetl/ai-meta#31

kadyapam committed Jun 2, 2026

wiki: noetl-arrow-flight-client 0.5.0 (Phase C2.4 mTLS identity)

kadyapam committed Jun 1, 2026

wiki: noetl-arrow-flight-client 0.4.0 (Phase C2.3 bearer token)

kadyapam committed Jun 1, 2026

wiki: noetl-arrow-flight-client 0.3.0 (Phase C2.2 TLS client)

kadyapam committed Jun 1, 2026

wiki: FlightResolver::connect example uses http:// (tonic scheme) The `grpc://...` URL in the example doesn't actually work — tonic's `Endpoint::from_shared` requires `http://` or `https://`; the `grpc://` scheme other Flight clients (Java, Python pyarrow) accept surfaces as `Bad :scheme header` at first request time. Matches the corrected `derive_flight_endpoint` in noetl-tools 2.12.0 and the doc comment fix in noetl/cli#44. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed Jun 1, 2026

docs(architecture): note get_flight_info (Phase C1) + result_fetch playbook surface Updates noetl-arrow-flight-client after Phase C1 landed (noetl/noetl#644 + noetl/cli#43, crates.io 0.2.0): - Workspace table row notes both releases (0.1.0 + 0.2.0) and links to the matching server PR. - "What it gives you" block now includes the get_flight_info recipe with the FlightInfoSummary field rundown. - New boundary-discipline paragraph cross-links to the `result_fetch` tool kind in noetl-tools as the playbook-policy layer that consumes this crate. - Phase scope updated: A ✅, B ✅ (crates.io 0.1.0), C1 ✅ (crates.io 0.2.0), playbook surface ✅, C2 + multi-endpoint sharding deferred.

kadyapam committed Jun 1, 2026

docs(architecture): document noetl-arrow-flight-client workspace crate (R-2.3 Phase B) Adds the new crate to the workspace-members table, extends the source-layout block, and adds a dedicated `## noetl-arrow-flight-client` section covering: - What it gives you (FlightResolver + resolve/resolve_rows API). - The Python server it pairs with (noetl/noetl#643, Phase A). - Boundary discipline (thin RPC client; scrub stays server-side). - Phase scope (A done, B done in noetl/cli#41, worker callsite deferred, Phase C planned). - Cross-links to the wiki page on noetl's Server API table (arrow_flight_result_fetch).

kadyapam committed Jun 1, 2026

executor-crate-architecture: add noetl-arrow-cache (R-2.1) PR noetl/cli#39 added the noetl-arrow-cache workspace crate — Rust mirror of Python's ArrowIpcSharedMemoryCache for same-node zero-copy IPC. Cut as 0.1.0 alongside noetl 4.2.0. Rewrite the page intro from a single-crate ("executor crate") framing to a workspace ("noetl-cli workspace crates") framing, add a workspace-member table, expand the source layout to include arrow-cache/, and document the arrow-cache public surface + cross-stack contract + data-plane role. Refs noetl/ai-meta#30 — Appendix H Rust migration umbrella.

kadyapam committed May 31, 2026

executor-crate-architecture: EE series complete (4 of 4) After noetl/worker#11 merged (noetl-worker 3.0.0), the four-PR cross-stack event envelope reconciliation series is done. All four producers / consumers (executor, Rust server, Python broker, worker) now emit + accept the same wire format on /api/events. Update the Deferred work bullet to mark PR-EE-3 ✅ with the noetl-worker version + PR link. Refs noetl/ai-meta#30.

kadyapam committed May 31, 2026

executor-crate-architecture: EE-2 + EE-4 landed, only EE-3 remains Update the Deferred work bullet to reflect the cross-stack event envelope reconciliation series progress after the noetl/server (Rust) + noetl/noetl (Python) PRs merged. - PR-EE-2 ✅ noetl/server#6 + pipeline-fix #7 (noetl-server 2.0.1) — EventRequest renamed name → event_type with alias; payload accepts context alias; added optional event_id / status / created_at. - PR-EE-4 ✅ noetl/noetl#639 (noetl 3.0.0) — EventEmitRequest accepts name / step / payload as validation aliases; worker_id lifted to top-level; EventType Literal → str (real finding: Literal was already dead code, production uses dot-notation event types throughout). - PR-EE-3 ⏳ noetl/worker — last step. Both servers now accept either form so the worker switch can land any time. Cross-link to noetl/server wiki event-envelope page.

kadyapam committed May 31, 2026

docs(executor): record PR-EE-1 (event envelope reconciliation begins) + tidy PR-2d-1 publish notes - Adds the PR-EE-1 row to the sub-PR landing-history table: ExecutorEvent enriched with event_id / worker_id / meta optional fields + payload serde alias, 0.3.0 -> 0.3.1, bin auto-bumped to noetl 4.1.0. - Removes the obsolete "pending publish" notes from the PR-2d-1 row now that 0.3.0 + noetl 4.0.0 are confirmed on crates.io. - Rewrites the Event envelope reconciliation deferred-work bullet with concrete progress: PR-EE-1 landed; PR-EE-2 (server Rust) + PR-EE-4 (Python) + PR-EE-3 (worker switch) remain. Worker switches last when both servers accept the new shape. Refs noetl/ai-meta#30 -- Appendix H umbrella. Refs noetl/cli#37 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 31, 2026

docs(executor): add PR-2d-2 row to worker adoption history Records the final R-1.2 worker-side sub-PR (noetl/worker#6). The Worker adoption history table now covers: - PR-2c (#2): condition surface; auto-bumped to 1.1.0, eventually reaching crates.io as 1.1.2 after pipeline fix. - PR-2d-2 (#6): CommandSource adoption; auto-bumped to 2.0.0 via feat!: prefix (major bump). Also fixes the PR-2c row's misleading "publish pending" note -- the worker release pipeline has been self-healing since noetl/worker#5 fixed the actions: write permission. Refs noetl/ai-meta#30 -- Appendix H umbrella. Refs noetl/worker#6 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 31, 2026

docs(executor): record PR-2d-1 — CommandSource trait redesign + design decisions - Adds the PR-2d-1 row to the noetl/cli sub-PR landing-history table. Documents the breaking trait redesign (ack lifecycle + 4-state ClaimOutcome + AckHandle associated type + enriched Command with render_context + attempts), the 0.3.0 bump, and the auto-cascading noetl 4.0.0 release. - Adds a dedicated "Design decisions" sub-section memorializing the four load-bearing API choices (Pulled<H> as struct not tuple; AckHandle as associated type not generic; error carrier as String not anyhow::Error; render_context + attempts #[serde(default)] for forward compat) plus the lifecycle invariant ("exactly one ack/nack per pulled handle, sequential per task"). Each row gives the rationale so future readers understand why these choices land where they do. - Updates the deferred-work section to mark the noetl-worker release pipeline gap as fixed (worker#4 + #5 — pipeline now self-healing as of noetl-worker 1.1.2). Refs noetl/ai-meta#30 -- Appendix H umbrella. Refs noetl/cli#35 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 31, 2026

docs(executor): R-1.2 publishing + worker adoption history Extends the executor-crate-architecture wiki page beyond R-1.1 complete with two new sections: 1. **R-1.2 sub-PR landing history** on noetl/cli: - PR-1 (#32): publish noetl-executor 0.1.0 to crates.io; fixes the broken release pipeline that had been failing for 3 consecutive runs (bin stuck at 2.17.1 vs 2.24.0). - PR-2a (#33): align execution_id to i64 across the crate; semver-major to 0.2.0; bin auto-bumped to noetl 3.0.0 since semantic-release picked up the breaking-change language. - PR-2b (#34): add structured Condition + 12-variant Operator + evaluate_structured_condition for worker adoption; minor bump to 0.2.1; bin auto-bumped to noetl 3.1.0. 2. **Worker adoption history** on noetl/worker: - PR-2c (noetl/worker#2): adopts noetl-executor = "0.2" dep; case_evaluator.rs 437 -> 344 LoC; condition primitives delegate to the executor; pull-loop control flow stays in the worker per § H.10. Tagged v1.1.0 but crates.io publish pending (worker's semantic-release.yml is missing the follow-up `gh workflow run release.yml` step). 3. **Deferred work** section: event envelope reconciliation (worker WorkerEvent ↔ server EventEmitRequest ↔ executor ExecutorEvent), GCS sink → object_store (R-3 scope), and the worker release pipeline gap (same bug noetl/cli had pre-PR-1). Refs noetl/ai-meta#30 -- Appendix H umbrella. Refs noetl/cli#32, noetl/cli#33, noetl/cli#34, noetl/worker#2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): R-1.1 complete -- PR-2d row + final stats Adds the PR-2d row to the sub-PR landing-history table. Wrap-up PR: end-to-end integration tests (12 new in `executor/tests/`), crate README, lib.rs module-list update. Closes noetl/cli#19. The "After PR-2c-8, the remaining R-1.1 work is PR-2d" paragraph becomes "R-1.1 is complete after PR-2d ships" with pointers to R-1.2/R-1.3 (worker adoption), R-2 (Arrow), R-3 (object_store including the GCS sink follow-up). Stats footer rewritten as a final tally: - playbook_runner.rs: -1,082 net cumulative - noetl-executor: 80 unit + 12 integration = 92 tests - Workspace: 174 passing (was 162) - Tool kinds wired: 4 / inline-with-helpers: 3 / unsupported: 1 Refs noetl/cli#19 -- closing it via PR #31. Refs noetl/cli#31 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): record PR-2c-8 -- Tool::Auth + Tool::Sink helpers Adds the PR-2c-8 row to the sub-PR landing-history table. Like PR-2c-7, this row documents a sub-PR shaped as "codify the decision + extract pure helpers" rather than "migrate the code": - Tool::Auth context-update side-effects extracted into auth_context_updates (Vec<(key, value)>) so callers apply the set_variable calls explicitly. Preserves the CLI's pre-PR-2c-8 variable-set ordering. - Tool::Sink format conversion extracted into format_sink_payload + json_to_csv. Sink targets (file/duckdb/gcs) stay inline per § H.10 since noetl-tools has no file/cloud-storage equivalents. - Both dispatch arms now bail loudly with helper-pointing messages. No semantic-divergence rows added — `format_sink_payload` is byte-for-byte equivalent to the inline impl; `json_to_csv` lifted verbatim; `auth_context_updates` preserves the original ordering. Stats footer bumped: - playbook_runner.rs: 1,636 -> 1,606 (-1,082 net cumulative) - noetl-executor unit tests: 65 -> 80 - workspace tests: 147 -> 162 - new code in noetl-executor: ~3,050 -> ~3,400 LoC Refs noetl/cli#19 -- R-1.1 sub-issue. Refs noetl/cli#30 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): record PR-2c-7 -- § H.10 codification for Tool::Playbook Adds the PR-2c-7 row to the sub-PR landing-history table. Unlike the previous five tool-replacement rows, this one DOESN'T migrate an inline implementation -- it encodes the § H.10 architectural finding that `Tool::Playbook` is not bridgeable: sub-playbook recursion is part of the CLI's tree walker by design. The bridge dispatch arm bails loudly with a § H.10-citing message instead of silently returning an empty outcome; the variable-preparation step (DSL v2 input / DSL v1 args merging) moves into a new `prepare_sub_playbook_vars` helper that's reusable and testable. No semantic-divergence rows added for this PR — no execution path changed. This is the first sub-PR in the series where the right shape is "codify the decision," not "migrate the code." Stats footer bumped: - playbook_runner.rs: 1,638 -> 1,636 (-1,052 net cumulative) - noetl-executor unit tests: 57 -> 65 (+8 for prepare_sub_playbook_vars + bail test) - workspace tests: 139 -> 147 - new code in noetl-executor: ~2,800 -> ~3,050 LoC Refs noetl/cli#19 -- R-1.1 sub-issue. Refs noetl/cli#29 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): record PR-2c-6 -- Tool::DuckDb via noetl-tools bridge Adds the PR-2c-6 row to the sub-PR landing-history table and four semantic-divergence rows under [#28]: - duckdb SELECT result envelope: noetl-tools' {columns, rows, row_count} reshapes back to CLI's rows-array shape - duckdb non-SELECT result envelope: {affected_rows: N} maps back to the CLI's literal {"status": "ok"} string - duckdb params: silently ignored -> bound as JSON at `?` placeholders (feature gain) - duckdb path resolution + mkdir: CLI keeps owning these at the call site (bridge has no knowledge of the playbook directory) Stats footer bumped: - playbook_runner.rs: 1,641 -> 1,638 (-1,050 net cumulative) - noetl-executor unit tests: 48 -> 57 - workspace tests: 130 -> 139 - new code in noetl-executor: ~2,550 -> ~2,800 LoC Refs noetl/cli#19 -- R-1.1 sub-issue. Refs noetl/cli#28 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): record PR-2c-5 -- Tool::Http via noetl-tools bridge Adds the PR-2c-5 row to the sub-PR landing-history table and four semantic-divergence rows under [#27]: - http transport: curl subprocess -> reqwest direct - http auth (GCP ADC): gcloud shellout -> gcp_auth crate - http JSON body: verbatim -> auto-detected (reqwest sets Content-Type) - http result envelope: pass-through reshape so 4xx/5xx come back inside the {status, body} envelope (not as anyhow errors) Plus a fifth row noting that Tool::Auth now shares the same GCP ADC code path via the new resolve_auth_to_bearer helper. Stats footer bumped: - playbook_runner.rs: 1,656 -> 1,641 (-1,047 net cumulative) - noetl-executor unit tests: 39 -> 48 - workspace tests: 80 -> 130 (was undercounting one binary) - new code in noetl-executor: ~2,250 -> ~2,550 LoC Refs noetl/cli#19 -- R-1.1 sub-issue. Refs noetl/cli#27 -- the PR this row documents. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 30, 2026

docs(executor): add PR-2c-4 row + shell semantic-divergence entries + bump stats - Adds PR-2c-4 (Tool::Shell wired) to sub-PR landing-history table. - New rows in 'Documented semantic divergences' table for the shell stdout streaming loss (real-time line-by-line -> end-of-execution burst) and the result-envelope unwrap. - Stats footer bumped: - playbook_runner.rs: 1,727 -> 1,656 - noetl-executor LoC: ~2,100 -> ~2,250 - noetl-executor unit tests: 34 -> 39 - Workspace tests: 75 -> 80 Refs noetl/cli#19 Refs noetl/cli#26

kadyapam committed May 30, 2026

docs(executor): add PR-2c-3 row + semantic-divergence table + bump stats - Adds PR-2c-3 (Tool::Rhai wired) to sub-PR landing-history table. - New 'Documented semantic divergences' table cross-references the PR for each behaviour delta: timestamp() shape change, HTTP curl->reqwest, get_gcp_token gcloud->gcp_auth. Subsequent per-tool replacements (PR-2c-4 shell, PR-2c-5 http, etc.) add rows to this table when they cross behaviour lines. - Stats footer bumped: - playbook_runner.rs: 1,964 -> 1,727 - noetl-executor LoC: ~1,850 -> ~2,100 - noetl-executor unit tests: 29 -> 34 - Workspace tests: 75 (34 noetl-executor + 41 noetl bin) Refs noetl/cli#19 Refs noetl/cli#25

kadyapam committed May 30, 2026

docs(executor): add PR-2c-2 to sub-PR landing history + bump stats Adds PR-2c-2 row (tools_bridge adapters) to the sub-PR table and updates the stats footer: - noetl-executor LoC: ~1,400 → ~1,850 - noetl-executor unit tests: 17 → 29 - Workspace tests: 41 (29 noetl-executor + 12 noetl bin) Refs noetl/cli#19

kadyapam committed May 30, 2026

docs: add noetl-executor workspace crate architecture page Documents the workspace-member crate introduced in v2.17.1+ via R-1.1 of the Rust migration roadmap (Appendix H of the global hybrid cloud blueprint). The page covers: - Source layout (executor/ as a workspace member of repos/cli) - What lives in noetl-executor (playbook types, template + condition + capabilities utilities, events shape, tools_bridge scaffold, worker-only CommandSource trait) and why - What does NOT live in noetl-executor (CLI tree walker, CLI inline tool implementations, worker NATS pull loop, RunOutcome envelope) - The § H.10 architectural finding (tree walker vs pull-model) that rescoped noetl-executor from a control-loop crate into a utilities-and-types crate during R-1.1 PR-2b - Cross-references to Appendix H, Polars endpoint, Arrow data plane, tracking issues - Sub-PR landing history: PR-1 (#20), PR-2a (#21), PR-2b (#22), PR-2c-1 (#23 open) - Stats: playbook_runner.rs 2,688 -> 1,964 lines; noetl-executor ~1,400 LoC; unit tests 0 -> 17 Sidebar gains a new "Architecture" section linking the page. Closes the wiki-maintenance Rule 1 gap that opened during PR-1 ("first touch of substantive surface ships with a wiki page") and continued through PR-2a / PR-2b / PR-2c-1. Each future sub-PR in this arc (PR-2c-2 onwards) extends the sub-PR landing history table and any module-level additions. Refs noetl/cli#19 Refs noetl/ai-meta#30

kadyapam committed May 30, 2026

wiki(cli): correct config file path — ~/.noetl/config.yaml (was .toml) The CLI persists context state as YAML at ``~/.noetl/config.yaml`` (per ``repos/cli/src/config.rs:105``), not TOML. Fixes all references across: - Home.md - context-model.md (including the storage-location diagram) - context-list-use-delete.md (``context use`` persistence) - global-context-flag.md (``--context`` read-but-not-write semantics) - auth-login.md (post-login state diagram + verify section) The Duffel smoke test against GKE this session surfaced the discrepancy: the awk-based extractor in operations/notes.md was pointing at the wrong file. Refs noetl/ai-meta#22

kadyapam committed May 28, 2026

wiki(cli): clarify auth login token caching — no export needed The "export NOETL_SESSION_TOKEN=..." line printed after a successful ``noetl auth login`` confuses first-time operators who think they must run the export for subsequent CLI calls to work. They don't: the CLI saves the token onto the context file (``Saved: context '<name>'``) and reads it from there. - auth-login.md: new "After a successful login — do I need to export the token?" section with verify steps, when the env var actually matters, and a diagram of state-after-login. - connecting-to-a-cluster.md: short callout pointing back to the full explanation so people working through the gateway path see it inline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 28, 2026

wiki(cli): document context model, init/update/port-forward, global --context, auth login PKCE hints, exit code 77 Covers the CLI changes shipped in noetl/cli v2.15.0 - v2.17.0 (PRs #13, #14, #16, #17): - context init --from-gateway: bootstrap a context by reading the gateway's GET /api/runtime/contract auth0 block, with operator confirmation + warning when no block is exposed. - context update: patch a context's fields in place; empty string clears Auth0 / kube fields without dropping the cached session token. - context port-forward: managed kubectl tunnel with --detach / --stop / --status, PID file at ~/.noetl/port-forwards/<ctx>.pid, TcpListener bind probe before spawn so stray kubectl from another shell fails fast. - noetl --context <name>: global per-command override mirroring kubectl --context. - auth login --browser-pkce pre-flight: prints the redirect URI + Auth0 dashboard URL so operators can verify the callback is allowlisted before the browser opens. - Exit code 77: dedicated "gateway session expired" code for scripts. Home.md rebuilt as a hub linking the per-command pages; _Sidebar.md adds nav. Adds connecting-to-a-cluster.md mirroring the three supported connection paths (gateway / port-forward / direct). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

kadyapam committed May 28, 2026

wiki(home): bootstrap with role + cross-links

kadyapam committed May 24, 2026

Initial Home page

kadyapam committed May 24, 2026