Skip to content

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 14 Nov 22:06
· 151 commits to main since this release
75268cd

Lightspeed v0.9.0 adds configuration as code for universe resources and access, an async Python SDK, live Platform updates, and environment gateway replicas. Declared machines keep stable environment IDs, session and bot lists gain stored summaries, and transcript submissions stay in order as responses arrive. The release also tightens API key authority, credential exposure, daemon file access and Platform defaults.

Compatibility and upgrade

  • This is a breaking release that requires a new Temporal namespace. scripts/release/workflow-compat.sh v0.8.0 reports breaking. Key attribution and workflow payloads changed; retained Temporal histories are not converted. Drain open work before upgrading and follow Upgrades and recovery.
  • Keep the runtime database. Migrations 16–24 advance it from revision 15 to 24, adding universe handles, stable API key IDs, declared environments, credential restrictions, model request headers, key attribution conversion, declared Platform configuration, session summaries and gateway ownership. No database reset is required when upgrading from v0.8.0. Migration 23 preserves existing activity state but does not backfill historical message previews or attachment summaries.
  • Platform schema revision advances from 4 to 5, storing declared universe configuration and verified directory-group claims. The API protocol identifier remains lightspeed.agent.api.v1, the envd protocol 5, and the controller protocol 4. These unchanged identifiers do not imply API or workflow compatibility: use matching v0.9.0 clients and components.
  • Runtime PostgreSQL connections require a direct connection or a session-mode pooler for change notifications and advisory-lock leadership. Transaction-mode pooling does not support these long-lived database sessions. Proxies serving Platform or SDK watches must pass text/event-stream responses without buffering.
  • Roll out the new daemon explicitly for its security fixes. Protocol-mismatch auto-upgrade will not select it because the envd protocol number is unchanged.

Moving from v0.8.0

  1. Before stopping the old release, record and remove extra headers on model providers; their old stored representation is not converted. Replace any literal api-key or x-api-key header on an MCP server with a token credential reference. Migration 20 refuses existing secret-bearing literal headers.

  2. Inventory environment bindings that use directSecret or a non-model provider credential. Migration 19 deletes direct-secret bindings. It preserves existing grant bindings by marking their grants retrievable; other provider bindings remain stored but will block environment execution until removed.

  3. Stop new submissions and connector ingress, drain or cancel open work, and stop all application writers and workers. Back up the runtime and Platform databases, object storage, and deployment secrets, including the unchanged runtime master key.

  4. Create a new Temporal namespace and set TEMPORAL_NAMESPACE on every runtime process, connector host and custom workflow worker. Do not overlap old and new workers on a task queue.

  5. Run the v0.9.0 executable against the existing runtime database:

    lightspeed-runtime migrate
    lightspeed-runtime schema-version

    Require runtime schema revision 24 before starting services. Migration 21 converts key attributions in records and session events. If the original key can no longer be identified, it retains the old prefix in an internal migration attribution. Old checkpoints with key attributions rebuild from the converted log; Temporal histories are not rewritten. Migration 24 clears old registered-daemon connection markers once because those rows have no gateway owner identity; daemons reconnect after startup.

  6. For multiple environment gateways, set each replica's LIGHTSPEED_ENVIRONMENT_GATEWAY_PEER_URL to its own directly reachable HTTP(S) origin and configure the same explicit LIGHTSPEED_ENVIRONMENT_GATEWAY_TOKEN on all replicas and workers. Workers keep the shared LIGHTSPEED_ENVIRONMENT_GATEWAY_URL. Keep /environment-gateway/peer/ and /environment-gateway/routes/ private; expose only the daemon connect and data paths publicly. A single gateway can omit the peer URL.

  7. Start matching components from one release manifest. Platform applies its own migrations to revision 5 on startup. Restore model headers using tagged literal or credential values. Replace deleted direct-secret bindings with retrievable credentials, and replace GitHub App private-key bindings with retrievable installation credentials. Add exposure: retrievable to configuration entries for bound grants. Exposure is fixed at creation; an unbound brokered grant needs a new retrievable credential to become bindable.

  8. Enable Bots and Channels under Settings → General → Features in universes that use them. Both now default to off where no choice was stored. The switches restrict Platform access; they do not stop runtime bots already running.

  9. Restart the stopped sessions you need, verify existing content and credentials, then reopen ingress. Bots resume in new sessions on their next event. Transcription jobs do not carry over; retries start new jobs. Review the upgrade guide for other open work lost at the namespace boundary.

Configuration as code and stable identities

  • lightspeed config export, diff, apply and schema manage YAML or JSON documents for universe resources and deployment API keys. Documents reference secrets in files or environment variables. Reapplying an unchanged document makes no writes; diff reports pending changes without applying them (#158).
  • Omitted sections and unnamed resources stay unmanaged. There is no resource pruning. An environment's explicit credential list does remove bindings omitted from that list; leaving the list out preserves them. Writes are individually repeatable, not one transaction across the document.
  • Universes have a required, deployment-unique handle, replacing slug. Existing valid names are preserved; missing or invalid ones receive a UUID-derived handle. A handle is a lowercase DNS label of up to 63 characters. UUIDs and resources stay unchanged.
  • Universe documents can declare displayName, members and directoryGroupMappings. Platform automatically adopts universes with a declared display name and synchronizes their names and declared access. Runtime-managed names are read-only in Platform; unnamed universes retain manual adoption (#167).
  • Declared members match verified email addresses or exact company-identity issuer/subject pairs. Directory-group mappings use verified issuer-qualified groups. Effective access is the highest applicable manual or declared role; removing a declaration preserves manual memberships. These grants do not create user accounts, grant platform administration, or change runtime API-key authority. Omitted access lists stay unmanaged; explicit empty lists clear declared grants.
  • lightspeed config apply --wait [SECONDS] waits for runtime discovery during startup, defaulting to 60 seconds when the flag has no value. It retries transient connection failures, rejects credentials and configuration errors immediately, and does not replay mutations with uncertain outcomes.
  • API keys have stable, scope-local keyId values that survive secret rotation. Keys, token credentials, providers and registration keys can be declared under chosen IDs with supplied secrets.
  • Fixed machines can be declared before their daemon connects. A persistent registration key can admit only declared names; releasing a bound daemon lets a replacement identity use the same environment and credential bindings.
  • The runtime image includes the matching lightspeed CLI at /usr/local/bin/lightspeed for administration and configuration jobs (#161).

Python SDK

  • The new lightspeed-ai client requires Python 3.12+, provides typed async gateway calls, session/run helpers, cursor-based event waiting, typed errors and timeline inspection. Python and TypeScript clients can continue the same sessions (#148).
  • A separate workflow module supplies Temporal helpers and decodable workflow types. Install the SDK from the release tag's clients/python directory; see the Python SDK guide.

Live lists, workspaces and transcripts

  • Session lists now include activity timing, the last run and its outcome, observed execution failures, a bounded message preview, the selected environment and attachment IDs. These summaries are maintained with event writes, so listing sessions does not replay their histories or load message blobs. Bot lists include display status and open/active session counts without querying Temporal (#169).
  • universe/watch streams session, bot, environment and workspace list-row changes over SSE on POST /rpc. TypeScript and Python clients provide watch() iterators with reconnection. The existing resource permissions and session visibility rules still apply. Notifications are lossy hints: refetch on watch/reset and retain periodic reads; there is no durable watch cursor.
  • Platform shares one watch connection across pages in the active universe. Lists update as resources change, and open workspace trees and saved file contents refresh after agent or browser edits. Unsaved drafts are preserved; saving is blocked against a changed revision until the draft is discarded and the latest file is loaded.
  • PostgreSQL change notifications wake event readers and resource waiters across runtime processes, with polling fallbacks for missed signals. The list watch complements the existing cursor-based session event reader; it does not replace transcript history.
  • Session input stays visible while the event stream catches up with its HTTP acknowledgement. Submission IDs match local messages to accepted runs and steering, keeping messages in place during confirmation. Steering stays inside its run from the first render, and acceptance events expose joins.submissionId for custom clients to make the same match.
  • TypeScript exports transcriptMetadata and transcriptAuthor to derive message times and author attribution from loaded history and live events. They distinguish authenticated actor attribution from display-only origin claims and handle partial or overlapping history. Name resolution and visual author/time labels remain application responsibilities.

Environment gateway replicas

  • Multiple environment-gateway processes can serve one shared address. PostgreSQL records which replica owns each daemon's control connection; replicas route stream setup to that owner and relay dial-backs to the replica holding the worker socket. A data stream crosses at most two gateway processes, regardless of fleet size. Workers keep one URL and the daemon wire protocol is unchanged.
  • Each gateway advertises its own peer URL and renews a 20-second lease every five seconds. Conditional ownership writes prevent a stale connection from disconnecting or updating its successor. Starting a replica leaves other replicas' connections intact; lease loss closes the affected gateway's sockets and makes its daemons reconnect. Lifecycle, display and registered sweeps each have one advisory-lock leader.
  • /ready reports whether the gateway accepts new work; /health remains process liveness. Shutdown refuses new routes and registrations, drains existing streams for up to ten seconds, then closes control sockets normally so daemons reconnect with their existing backoff. Peer endpoints require the shared deployment token, and signed, expiring pairing tokens remain one-time.
  • Failover restores connectivity but does not migrate existing streams or blindly repeat tool operations. Interrupted process waits preserve their handle for continuation; durable jobs retry with stable request IDs. A lost process-start acknowledgement or interrupted filesystem mutation can still have an uncertain outcome. See Networking and ingress for replica configuration.

API and CLI changes

  • deployment/universes/configuration/put manages declared display names and Platform access. session/list, bots/list and bots/read expose the new summary/status fields. Use the SDK watch() helpers for universe/watch; ordinary call() expects a single JSON response.
  • Replace universe slug fields and deployment/universes/slug/put with handle and deployment/universes/handle/put; the CLI uses universe set-handle. Key rotation and revocation now identify keys by keyId and scope rather than their secret prefix. Key attributions expose keyId and scopeKind instead of prefix.
  • Key creation and declaration require explicit groups: a non-empty list or "all". CLI scripts must pass --group or --all-groups. Stored keys keep their existing groups. api-key provision no longer prints a secret supplied through LIGHTSPEED_BOOTSTRAP_API_KEY.
  • auth/grants/put and auth/providers/put replace the former replace methods. Update callers to the matching contract. Snapshot commands move from vfs to workspace snapshot upload and workspace snapshot download; review CLI help when updating automation.
  • Model endpoint headers now use { type: "literal", value: "…" } or { type: "credential", grantId: "…" }. Headers named api-key, x-api-key, and model proxy-authorization require credential references. Use --header-credential in CLI scripts.
  • Unauthenticated /rpc calls in authenticated mode receive HTTP 401 before their bodies are read, with an unauthenticated JSON-RPC error and null ID. Authentication-store failures return HTTP 500. This release's SDKs handle the 401 as a typed error.
  • Trigger views no longer expose ingest paths or pairing codes. Authorized managers use bots/triggers/secrets/read; creation responses return secrets separately. Bot trigger tools no longer list those secrets.

Security, operations and fixes

  • Environment bindings accept retrievable grants or model API keys only; raw stored secrets and other provider credentials are refused. Credential-backed request headers resolve current token values for each request.
  • MCP OAuth discovery, registration, exchange and refresh follow LIGHTSPEED_MCP_PRIVATE_NETWORKS, including resolved address ranges. Replace the removed LIGHTSPEED_MCP_OAUTH_ALLOW_PRIVATE_NETWORKS switch with explicit entries for internal MCP and authorization servers.
  • Daemon file tools block the daemon state directory, registration key file and its mounted volume, and Linux /proc and /sys, including alternate paths and links. This restriction does not constrain commands started on the machine. The daemon no longer loads .env; pass its settings through the process environment or command line.
  • Runtime and connector Temporal connections support TLS, mutual TLS and API-key authentication. Use a host-and-port TEMPORAL_ADDRESS and explicit TEMPORAL_TLS* / TEMPORAL_API_KEY* settings.
  • Prompt loading and skill discovery require explicit switches in profile/session editors. Skill metadata is bounded and rejects control characters. Remote Markdown images render as links instead of loading automatically.
  • Platform reports the HTTP status and readable response text for non-JSON failures, including proxy timeouts, instead of a JSON parse error (#155).
  • OpenAI Responses requests now set store: false, fixing HTTP 409 failures when a later run reuses a compaction item. TLS and async runtime dependencies receive patch updates (#165).

Full changelog: v0.8.0...v0.9.0