Skip to content

feat(sdk): shared SDK core and TypeScript binding (RFC 0005)#1617

Draft
maxdubrinsky wants to merge 6 commits into
mainfrom
md/sdk-extraction-and-pi-extension
Draft

feat(sdk): shared SDK core and TypeScript binding (RFC 0005)#1617
maxdubrinsky wants to merge 6 commits into
mainfrom
md/sdk-extraction-and-pi-extension

Conversation

@maxdubrinsky
Copy link
Copy Markdown
Collaborator

Summary

Opens RFC 0005 for review with a prototype implementation and a demo. The RFC proposes extracting OpenShell's gRPC client, TLS, OIDC, and edge-tunnel plumbing out of openshell-cli into a shared openshell-sdk Rust crate, with openshell-sdk-node (published as @openshell/sdk) as the first language binding (napi-rs). The prototype refactors openshell-cli and openshell-tui onto the SDK; the demo is a Pi (pi.dev) coding-agent extension that wraps @openshell/sdk and treats OpenShell sandboxes as disposable sub-agents. Open as a draft so the implementation can move with RFC feedback.

Related Issue

Part of #1044 (Python and TypeScript SDK Support). Delivers the TypeScript half. The pure-Python SDK is unchanged (RFC 0005 non-goal).

Changes

  • RFC 0005 (rfc/0005-shared-sdk-core-and-ts-binding/README.md): motivation, surface area, error model, and the path for additional language bindings. State: review.
  • openshell-sdk crate: typed gRPC client, TLS resolver, OidcRefresher with single-flight semantics, edge-tunnel dialer, and a sandbox/exec API surface. crates/openshell-core/src/auth.rs and crates/openshell-cli/src/edge_tunnel.rs move into this crate.
  • openshell-sdk-node crate: napi-rs wrapper, published as @openshell/sdk. Generated index.d.ts / index.js are committed; per-platform .node binaries are rebuilt with napi build and gitignored. lib.mjs provides an ESM facade with an errorCode() helper for typed error discrimination.
  • CLI / TUI: openshell-cli/src/{tls.rs,oidc_auth.rs} shrink to thin wrappers. openshell-cli and openshell-tui consume the SDK directly. Consumer dependencies (tokio-rustls, tokio-tungstenite, tower) move into the SDK.
  • Demo (examples/pi-extension-sandbox-agents/): Pi extension registering five tools (openshell_run_task, openshell_spawn_sandbox, openshell_exec, openshell_list_sandboxes, openshell_destroy_sandbox) plus /openshell-health. openshell_run_task is the primary dispatch surface: create -> wait -> exec -> return -> delete (unless keep_sandbox=true).
  • CI: new sdk-node:check mise task and a TypeScript SDK job in branch-checks.yml. Rebuilds the napi binding, fails if regenerated index.{js,d.ts} differ from HEAD, then runs the smoke suite. Wired into [ci] so mise run pre-commit covers it. crates/openshell-sdk-node/package-lock.json is committed for deterministic npm ci.
  • Manual OIDC smoke: mise run oidc:smoke runs scripts/openshell-sdk-oidc-smoke.sh against a local Keycloak (started by mise run keycloak), exercising both openshell_sdk::oidc::{discover, refresh_token} and @openshell/sdk's OidcRefresher.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated: 3 unit + 10 mock-gateway integration tests in openshell-sdk; 5 smoke tests in openshell-sdk-node covering module exports, typed connect errors, and OidcRefresher single-flight semantics.
  • E2E tests not applicable. The Pi demo documents its own end-to-end exercise against a running gateway; the OIDC smoke covers the SDK's client-side auth path.

Manual smoke:

  • mise run keycloak && mise run oidc:smoke against a fresh local Keycloak (openshell realm). Rust SDK discover() + refresh_token() returned a 1230-byte JWT with a rotated refresh token and populated expires_at. The napi OidcRefresher collapsed 5 concurrent refresh() calls on an expired token into 1 token endpoint hit. A follow-up refresh() on the cached non-expired token short-circuited. Callback rejection surfaced as an auth error.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable): deferred. Will add architecture/sdk.md once RFC 0005 is accepted.

Deferred (explicit follow-ups)

  • architecture/sdk.md after RFC 0005 is accepted.
  • PyO3 binding to retire the hand-written Python SDK (separate RFC if scope grows).
  • Wiring the OIDC smoke into branch-e2e.yml with a Keycloak service container.

Captures the design behind extracting the shared client core out of
openshell-cli into a standalone openshell-sdk crate, plus the napi-rs
TypeScript binding (openshell-sdk-node, published as @openshell/sdk).

Covers motivation (CLI/TUI/embedders sharing one transport, OIDC, and
edge-tunnel implementation), surface area, error model, and the path
for future language bindings.
Per RFC 0005, lift the gRPC client, TLS, OIDC, edge-tunnel, and refresh
plumbing out of openshell-cli into a new openshell-sdk crate. CLI and
TUI now consume the SDK; openshell-cli/src/{tls.rs,oidc_auth.rs} shrink
to thin wrappers over the SDK's transport and OIDC modules.

- New crate openshell-sdk exposes a typed gRPC client, TLS resolver,
  OidcRefresher with single-flight semantics, edge-tunnel dialer, and a
  Sandbox-API surface that mirrors the existing CLI behavior.
- crates/openshell-core/src/auth.rs moves into the SDK as auth.rs.
- crates/openshell-cli/src/edge_tunnel.rs moves into the SDK as
  edge_tunnel.rs.

Tests: 3 unit + 10 mock-gateway integration tests in openshell-sdk.
Per RFC 0005, ship openshell-sdk-node as a napi-rs wrapper over the
openshell-sdk crate, exposing the same surface to TypeScript/Node
consumers as @openshell/sdk.

- New crate openshell-sdk-node binds OpenShellClient, OidcRefresher,
  the edge-tunnel dialer, and the sandbox/exec API through napi-rs.
- Generated index.d.ts and index.js are committed; the per-platform
  .node binary is gitignored and rebuilt with `napi build` per host.
- lib.mjs provides a small ESM facade with named exports plus the
  errorCode() helper for typed error discrimination.

Tests: 5-case smoke suite that exercises exports, typed connect errors,
and OidcRefresher single-flight semantics.
Adds a Pi (pi.dev) coding-agent extension under examples/ that treats
OpenShell sandboxes as disposable sub-agents. The extension wraps
@openshell/sdk and registers five tools (openshell_run_task,
openshell_spawn_sandbox, openshell_exec, openshell_list_sandboxes,
openshell_destroy_sandbox) plus a /openshell-health slash command.

openshell_run_task is the primary dispatch surface: one call creates a
sandbox, waits for ready, runs a command, streams stdout/stderr back as
the tool result, and deletes the sandbox unless keep_sandbox=true.
Without this, edits to crates/openshell-sdk-node/src/lib.rs that change
the napi surface would silently land with stale index.js / index.d.ts,
shipping a TypeScript SDK whose types lie about the runtime.

- New mise tasks: sdk-node:install, sdk-node:build, sdk-node:smoke, and
  sdk-node:check. The :check task rebuilds the binding, fails if the
  regenerated index.{js,d.ts} differ from HEAD, then runs the smoke
  suite.
- Wire sdk-node:check into [ci] so `mise run pre-commit` covers it.
- New "TypeScript SDK" job in branch-checks.yml runs sdk-node:check on
  linux-amd64 (the generated files are platform-agnostic; one arch is
  sufficient to catch drift).
- Commit crates/openshell-sdk-node/package-lock.json so `npm ci` in
  the new task is deterministic.
Provides a manual end-to-end check for the SDK's OIDC paths against a
live OIDC issuer. The existing Python e2e suite covers OIDC enforcement
on the gateway side; this covers the SDK as a client.

- scripts/openshell-sdk-oidc-smoke.sh — orchestrates: verifies the
  Keycloak realm is reachable, mints an initial refresh token via the
  password grant, then runs the Rust and TS smoke binaries in sequence.
- crates/openshell-sdk/examples/oidc_smoke.rs — exercises
  openshell_sdk::oidc::{discover, refresh_token} against the live
  issuer.
- crates/openshell-sdk-node/test/oidc_smoke.mjs — drives the napi
  OidcRefresher with a JS callback that hits the live token endpoint,
  validates single-flight under load (5 concurrent calls → 1 hit) and
  callback rejection mapping.
- tasks/keycloak.toml — `mise run oidc:smoke` runs the script after
  `mise run keycloak` has started a local instance.

Run order:
    mise run keycloak     # docker container, port 8180
    mise run oidc:smoke   # builds the napi binding, runs both smokes
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 28, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

zanetworker added a commit to zanetworker/OpenShell that referenced this pull request May 29, 2026
- Decouple OIDC motivation from K8s framing per @derekwaynecarr feedback
- Reframe as certificate distribution problem, not namespace problem
- Add relationship section positioning RFC 0006 relative to RFC 0005
- Add links to RFC 0005 (NVIDIA#1617), Python OIDC PR (NVIDIA#1621), roadmap (NVIDIA#1044)
@mrunalp
Copy link
Copy Markdown
Collaborator

mrunalp commented May 29, 2026

I mainly looked at the RFC doc and I like the proposed refactoring 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants