Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,18 @@ a database request, a feature rollout, access policy, or a business workflow as
same live, conflict-aware editing experience should work wherever a resource expresses intent and a
controller reports what became true.

## Why a gateway

Kubernetes already has a good change feed: a watch, documented under
[efficient detection of changes](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes).
A browser cannot use it directly. Watching requires a cluster credential, the API server serves no
CORS, and a watch hands back whole objects including `Secret` data. The gateway holds the credential,
withholds what the browser should not see, and re-frames the stream as SSE that `EventSource` reads
natively. It also shares one upstream watch per scope, so ten tabs are not ten watches on the API
server.

[Why a gateway](docs/why-a-gateway.md) works through this in full.

## How it fits

```mermaid
Expand Down Expand Up @@ -117,11 +129,14 @@ browser. Use [`gateway.ValidateMergePatch`](gateway/patch.go) in the host save h

## Guides

- [Glossary for frontend developers](docs/glossary.md): the Kubernetes vocabulary you actually need, and where each word shows up in the library.
- [Why a gateway](docs/why-a-gateway.md): why the browser cannot watch the API server, and why watches are shared.
- [Adopting krm-stream](docs/adopting.md): same-origin cookie, bearer-token, and shared-watch setups.
- [Authentication and authorization](docs/auth.md): identity and RBAC boundaries.
- [Saving edits safely](docs/saving.md): patch validation and host write responsibilities.
- [Operating krm-stream](docs/operations.md): metrics, alerts, and runtime controls.
- [Client state model](docs/client-state-model.md): drafts, conflicts, redactions, and keyed lists.
- [Alternatives and prior art](docs/alternatives.md): how this differs from Kubernetes clients, browser dashboards, and config-as-data systems.
- [Releasing](docs/releasing.md): release workflow and publication prerequisites.

## Requirements and maturity
Expand Down
109 changes: 109 additions & 0 deletions docs/alternatives.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Alternatives and prior art

Where krm-stream sits relative to existing work, and what it does not try to be.

krm-stream is two things: a wire contract for streaming a scoped, redacted projection of KRM
resources into a browser, and a client store that keeps server truth and local drafts separate so a
user can keep typing while the cluster changes underneath them. Most neighbouring projects solve one
half and leave the other to the application.

## Kubernetes client libraries

**[@kubernetes/client-node](https://github.com/kubernetes-client/javascript)** is the official
JavaScript client. It covers watches and informers, including the parts that are easy to get wrong:
`resourceVersion` bookkeeping, bookmarks, `410 Gone` and relist. It is built for Node, speaks
Kubernetes API concepts directly, and ships credential and kubeconfig handling that does not belong
in a browser bundle. It has no notion of a draft, a conflict, or a redacted projection. The
krm-stream gateway sits on this class of library rather than replacing it.

**[kube-watch](https://github.com/subk/kube-watch)** and similar wrappers are the same story with
less coverage: an event emitter over the watch verb, server-side.

**[Raw Kubernetes watch](https://kubernetes.io/docs/reference/using-api/api-concepts/)** gives you
`ADDED`, `MODIFIED`, `DELETED`, bookmarks and streaming initial events. It is the machinery
underneath everything here, not a browser-facing contract. A client still has to solve reconnect,
snapshot completion, history gaps and reconciliation with local state itself. That is the work
krm-stream packages up.

## Browser Kubernetes UIs

**[Headlamp](https://headlamp.dev/)** is the closest architectural precedent for the gateway. Its
browser opens a single WebSocket to `headlamp-server`, which fans out to the cluster API servers.
That is the same posture krm-stream takes with SSE: the API server is never exposed to the browser.
Headlamp also exposes TypeScript APIs (`apiProxy`, `streamResults`, object hooks) to plugin authors.
The differences are that it is a Kubernetes UI and plugin host, its client APIs are React-shaped and
coupled to the Headlamp runtime, and its streaming model is watch-and-replace. Editing is a YAML
editor with `resourceVersion` optimistic concurrency, not a draft that survives a concurrent server
change. If you want a Kubernetes dashboard, use Headlamp. krm-stream is for embedding KRM-backed
live state in an application that is not a Kubernetes UI.

**[@hawtio/kubernetes-api](https://github.com/hawtio/hawtio-kubernetes-api)** is the historical
precedent: browser-side Angular client, WebSocket watch, in-memory collections, CRUD. It talks
directly to the API server and needs CORS configured on the cluster. Its model is to watch and
replace the collection. No drafts, no three-way merge.

**Lens, Skooner, and the Kubernetes Dashboard** are applications, not libraries. Nothing reusable is
published for embedding.

## Config-as-data systems

These share the premise that configuration is data, queryable and mutable through an API, rather
than templates to be rendered. They operate at the package and delivery layer rather than the
live-editing layer.

**[kpt](https://kpt.dev/guides/rationale/) and [Porch](https://github.com/kptdev/porch)** are the
reference Configuration-as-Data implementation, and where the term comes from: configuration data is
the source of truth, stored separately from live state, with the code that acts on it kept out of
the data. They manage the lifecycle of KRM packages in Git, from Draft through Proposed to
Published, with KRM functions mutating packages.

The vocabulary overlaps. Porch has drafts too, but a Porch draft is a package revision moving
through approval gates over minutes or days, not a form field a user is holding while a controller
updates `.status`. Porch is asynchronous and Git-backed. krm-stream is sub-second and
cluster-backed.

**[gitops-reverser](https://reversegitops.dev)** is a complement, not an alternative, and it is why
the krm-stream save boundary looks the way it does. Reverse GitOps puts an API in front and lets Git
remember: a validated write lands on a user-facing CRD, and the accepted intent is recorded to Git
as a manifest, with the actor as commit author, for Flux or Argo CD to distribute. krm-stream is the
read and edit half of that loop. It streams those CRDs into a browser and produces an RFC 7386 merge
patch when the user saves. The two meet at the API: krm-stream never writes, it hands the host a
patch to validate, and the host's write is what reverse GitOps records.

## Local-first and merge libraries

Automerge, Yjs, and the CRDT family solve concurrent editing, and generic JSON-merge libraries solve
structural merging. None of them know what a `resourceVersion` is, that `spec.containers` is keyed
by `name` rather than by index, that a snapshot has a completion point, or that a redacted field
must not be sent back on save. They are ingredients, not alternatives.

The krm-stream merge is deliberately not a CRDT. KRM has one authoritative writer, the API server,
so last-write-wins with the conflict surfaced to the user is the model that matches the data.

**TanStack Query, SWR and Apollo** are the closest analogue in application code: a server cache plus
optimistic updates. They have no streaming Kubernetes source and no snapshot semantics, and their
optimistic update is discarded on refetch, which is the failure krm-stream exists to prevent.

## Summary

| | Streams KRM to browser | Browser-safe (no direct API server) | Framework-independent | Server truth vs. local draft | Conflict-aware three-way merge |
|---|---|---|---|---|---|
| [krm-stream](https://github.com/ConfigButler/krm-stream) | yes | yes (gateway) | yes | yes | yes |
| [@kubernetes/client-node](https://github.com/kubernetes-client/javascript) | no (Node only) | n/a | yes | no | no |
| [Headlamp](https://headlamp.dev/) | yes | yes (headlamp-server) | no (React/plugin host) | no | no |
| [@hawtio/kubernetes-api](https://github.com/hawtio/hawtio-kubernetes-api) | yes | no (CORS to API server) | no (Angular) | no | no |
| [kpt](https://kpt.dev/guides/rationale/) / [Porch](https://github.com/kptdev/porch) | no | n/a | n/a | package drafts, not field drafts | no |
| [Automerge](https://automerge.org/) / [Yjs](https://yjs.dev/) and merge libs | no | n/a | yes | yes | yes, but KRM-unaware |

[gitops-reverser](https://reversegitops.dev) is absent from that table on purpose. It is the write
and record half of the same loop, not a competing way to do this half.

## The claim we can defend

Not "the first Kubernetes streaming library". Watch clients and browser dashboards have existed for
years, and the gateway stands on them.

What is new is the combination: a library for conflict-aware live editing of KRM resources in
browser applications, independent of any UI framework. It is the state layer between Kubernetes
client libraries and application form state. Stated more cautiously: to our knowledge, the first
open browser client and gateway designed for that.
123 changes: 123 additions & 0 deletions docs/glossary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# Glossary for frontend developers

You do not need to know Kubernetes to use krm-stream. You do need about a dozen words. This page
defines them, then shows where each one appears in the library.

## The data

**KRM (Kubernetes Resource Model)** is the shape every object here has. It is JSON with four
conventions: `apiVersion`, `kind`, `metadata`, and a body. Read it as a schema for a declarative
object with an identity and a desired state. Nothing about it is specific to containers. A
`Database`, a `FeatureFlag` or a `Tenant` can be a KRM resource.

**Resource** is one such object. Identified by `apiVersion`, `kind`, `namespace` and `name`, and
uniquely by `metadata.uid`. The store keys on `uid` rather than name, because a delete and recreate
under the same name is a different object and must not inherit the old draft.

**CRD (Custom Resource Definition)** is how a team adds their own `kind`. It is why KRM works as an
application configuration API and not only as cluster plumbing: your product's domain objects can be
resources.

**`spec` and `status`** are the split to remember. `spec` is what a human or agent wants. `status`
is what the system observed. Users edit `spec`. Nobody edits `status`, and the store enforces that:
`status` is read-only and never part of a save.

**`resourceVersion`** is an opaque server-assigned token that changes on every write. It works like
an ETag, and it is how the server detects that you edited a stale copy.

**Namespace** is a folder. Resources live in one, and some kinds are cluster-wide instead.

## The stream

**Watch** is the Kubernetes primitive for "tell me when this changes". It yields a stream of `ADDED`,
`MODIFIED` and `DELETED` events. It is server-side and stateful, and it has edge cases: history
expiry, relist, bookmarks. The gateway handles those. You do not see them.

**Snapshot** is the initial run of events describing the world as it currently is, before live
changes arrive. It has a completion point, the `synced` event. Until a snapshot completes, the client
cannot tell whether a resource it remembers is gone or simply not re-sent yet, so an incomplete
snapshot never prunes state.

**SSE (Server-Sent Events)** is the browser transport: an HTTP response that stays open and streams
text events. `EventSource` is built into every browser. It is one-directional, which is all a read
stream needs.

**Gateway** is the server-side piece you mount in your own Go application. It holds the Kubernetes
credentials, decides who may see what, and turns a watch into a scoped SSE stream. The browser never
receives a cluster credential or an API-server URL.

**Projection** is the subset of a resource the gateway sends. What the browser receives may be less
than what exists upstream.

**Redaction** is a path the gateway knows exists but withholds, such as a Secret value. It is not
sent as a placeholder; it is absent from the object, and listed in `redactions(id)` so the UI can
render it as withheld and offer no editor. It follows that a redacted field must never be written
back, or the browser would erase it.

## The editing

**Draft** is the object your form is bound to. It is separate from the server object, and the stream
cannot overwrite it without telling you.

**Three-way merge**: while you are editing a resource, new server updates can arrive. A three-way
merge uses the version you started editing, your draft, and the new server version to combine
non-conflicting changes without overwriting your work.

At each editable path the store asks two questions: did the server change this field, and did you
change this field?

| Server changed it | You changed it | Result |
|---|---|---|
| yes | no | the server value flows into your draft |
| no | yes | your edit stands |
| yes | yes, to the same value | converge, no conflict |
| yes | yes, to a different value | your draft stands, and a conflict is recorded |

Only the last row needs a human. Without this, a controller updating one annotation would discard
the text you were typing in an unrelated field.

**Conflict** is the fourth row above. It is not an error and it does not block a save. The draft
still wins, and `conflicts(id)` gives the UI what it needs to show the server value alongside it,
with `takeTheirs` or `revert` as the ways out.

**Associative list** is a Kubernetes array that behaves as a map. `spec.containers` is keyed by
`name`, not by index. A merge that treats it as an array corrupts it when two people change
different containers. The store merges these by key.

**RFC 7386 merge patch** is the save format: a JSON document containing only what changed, where
`null` means delete. The store builds it by diffing draft against server over editable paths only.
It never diffs the whole projected object, which would turn a field that is absent because of a
projection into a deletion.

## How this hooks into krm-stream

The read path, in the order the words appear:

1. Your Go application mounts the **gateway**. It authenticates the user, decides the **scope**, and
opens a **watch**.
2. The gateway applies a **projection** and its **redactions**, then streams a **snapshot** over
**SSE**, followed by live updates.
3. `LiveResourceStore` consumes those events and keeps, per **`uid`**: `server(id)` for the server
object, `draft(id)` for yours, plus `conflicts(id)` and `redactions(id)`.
4. Every incoming update runs the **three-way merge** over `server` as the base you started from,
`draft` as what you typed, and the new server object. Server changes you did not touch appear in
the form. Your edits survive. Collisions land in `conflicts(id)`.

The write path is not the library's:

5. On save, `patch(id)` returns an **RFC 7386 merge patch**, or `null` when nothing changed.
6. You send that to your own save endpoint. The store never writes to Kubernetes.
7. Your handler calls [`gateway.ValidateMergePatch`](../gateway/patch.go), which rejects a patch
touching anything the effective projection withheld or stripped: a redacted path,
`metadata.managedFields`, the last-applied annotation, and `status` under `ProjectionSpec`. It is
what stops a buggy or hostile browser from destroying what it was never shown. Do not skip it on
the grounds that the store is careful, because the store runs on the caller's machine.
8. The write goes to the API. The watch sees it, it returns down the stream as an ordinary update,
and the merge converges your draft with it. Your own write needs no special handling.

If you know TanStack Query or SWR, this is the same server cache with local edits, with two
differences: the cache is pushed rather than refetched, and the local edit is not discarded when new
server data arrives.

Next: [Client state model](client-state-model.md) for the API surface, and
[Saving edits safely](saving.md) for the host's responsibilities on the write path.
64 changes: 64 additions & 0 deletions docs/why-a-gateway.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Why a gateway

Why krm-stream puts a server between the browser and the Kubernetes API, rather than letting the
browser watch the API server itself.

## The mechanism it builds on

Kubernetes has an efficient change feed. A `GET` with `?watch=1` streams `ADDED`, `MODIFIED` and
`DELETED` events for a resource, with `resourceVersion` as the position in the stream, bookmarks to
keep that position cheap, and `410 Gone` when the position has aged out of the server's cache. It is
documented under
[efficient detection of changes](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes),
and it is what the gateway consumes upstream. Nothing here replaces it.

## Why the browser cannot use it directly

Not for transport reasons. A watch is an ordinary chunked HTTP response carrying newline-delimited
JSON. It is not a protocol upgrade; upgrades are what `exec`, `attach` and `port-forward` need, and a
watch is not one of them. The obstacles are around the stream rather than in it.

**It needs a cluster credential.** A watch means presenting a bearer token or a client certificate
that Kubernetes RBAC recognises. Shipping either to a browser gives every tab, and anything running
in it, an identity in your cluster. There is no restriction you can attach in the browser that the
browser cannot also remove.

**`EventSource` cannot read it.** A watch is newline-delimited JSON, not SSE framing, and
`EventSource` cannot set an `Authorization` header. Consuming a watch in a browser means `fetch`, a
`ReadableStream`, and your own reconnection and resume logic, which returns you to the credential
problem with more code around it.

**The API server serves no CORS.** Reaching it cross-origin requires `--cors-allowed-origins` set
cluster-wide on the API server, for your web application. That is the concession
`@hawtio/kubernetes-api` required (see [alternatives](alternatives.md)), and most operators will not
make it.

**The browser would see whole objects.** A watch returns everything: `Secret` data, `managedFields`,
`status`, fields belonging to other tenants of the same namespace. Withholding those has to happen
somewhere the user does not control, which means the server.

The gateway is that server. It holds the credential, applies the projection and its redactions,
enforces the scope, and re-frames the result as SSE, which the browser reads natively with no bundler
and no reconnection logic in your application.

## Why watches are shared

A watch is not free upstream. Each one is a connection and a registered watcher on the API server,
and it delivers every event in its scope. Ten tabs on the same namespace, watching directly, are ten
watches, ten snapshots and ten copies of the same object graph. Close a floor of laptop lids and
reopen them and the reconnect storm arrives at the API server multiplied by the number of tabs.

[`gateway.SharedBackend`](../gateway/shared.go) opens one upstream watch per scope rather than per
tab, and serves every subscriber from its cache. A tab joining a scope that is already open gets its
`reset`…`synced` snapshot from that warm cache without reaching the API server at all.

Sharing is opt-in, and the reason is a real trade. A shared watch can be opened only once, so it can
be opened as only one identity: your service account. Without sharing, the client acts as the caller
and Kubernetes RBAC is the enforcement, so no bug in this library can hand a caller an object they
may not see. With sharing, your `Authorizer` becomes the only thing between a caller and the cache.

There is a way to take the fan-out without giving up the boundary. Pair `SharedBackend` with
[`kube.SSARAuthorizer`](../gateway/kube/authz.go), which asks the API server, through a
SubjectAccessReview, whether this user may list and watch this resource here, before the subscriber
is served from the shared cache. Kubernetes decides again, per user, per snapshot cycle, and the
sharing costs one round-trip. Read [auth.md](auth.md) before wiring it.
Loading