Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,9 @@ SDK-compat coverage across AWS, Azure, and GCP:
The Kubernetes story is two layers, both shipped:

- **Control plane** (EKS / AKS / GKE) — cluster, node-pool, addon / Fargate / maintenance-config lifecycle via the real cloud SDKs.
- **Data plane** (in-memory Kubernetes API) — Namespace, Pod, Service, ConfigMap, Secret, ServiceAccount, Deployment, Endpoints. Supports CRUD + JSON-merge Patch + Watch streaming, so real `client-go` `Informer`/`Reflector` machinery works against a cloudemu-emulated cluster. Kubeconfigs returned by the control plane point at the in-memory data plane — `kubectl apply -f deployment.yaml` followed by `kubectl get pods` round-trips end-to-end.
- **Data plane** (in-memory Kubernetes API) — core, apps, batch, networking, rbac, storage, autoscaling, policy, discovery, **apiextensions** (CRDs) and **admissionregistration** kinds. Supports CRUD, all patch types + **server-side apply** (field ownership + conflicts), `?dryRun=All`, finalizers, `limit`/`continue` pagination, watch streaming with `resourceVersion` resume + BOOKMARK — so real `client-go` `Informer`/`Reflector` machinery works against a cloudemu-emulated cluster. Kubeconfigs returned by the control plane point at the in-memory data plane — `kubectl apply -f deployment.yaml` followed by `kubectl get pods` round-trips end-to-end.

What's intentionally out of scope: real controllers (Deployment ReplicaSet ↛ Pod), scheduler (Pods stay Pending), RBAC, PV/PVC, StatefulSet/DaemonSet/Job/CronJob, Ingress.
Emulation model: there is no scheduler or kubelet, so controllers converge **synchronously** — a Deployment interposes a ReplicaSet and materializes Pods straight to Running (a Job's straight to Succeeded), and Services get Endpoints, on every write. On top of the raw object store it also serves **CRDs** (dynamic servable kinds), **`metrics.k8s.io`** + **HPA** actuation, object-count **ResourceQuota** / **LimitRange** / **PDB-gated eviction** enforcement, **RBAC** SubjectAccessReview + **NetworkPolicy** evaluation, and **opt-in admission webhooks**. See [docs/services.md](docs/services.md) §18 for the authoritative capability list.

Full per-service operation list: [docs/services.md](docs/services.md).
Per-handler protocol details and limitations: [docs/sdk-server.md](docs/sdk-server.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/sdk-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ Registration order matters when handlers share a path prefix — `awsserver.New`

Kubernetes ships as **two cooperating handlers**: per-provider control planes (EKS / AKS / GKE — clusters + node pools + addons / Fargate / maintenance configs) and a shared in-memory **data plane** registered under `/k8s/{cluster-uid}/`. The control plane mints a UID on every cluster Create and embeds it in the kubeconfig (or `Cluster.Endpoint` for GKE) along with a CA that certifies the data-plane serving cert, so `client-go` and `kubectl` connect over **validated TLS**. The data plane behaves like a tiny always-converged cluster (minikube-like): a synchronous reconcile engine runs on every write, so Deployments/ReplicaSets/StatefulSets/DaemonSets materialize **Running** Pods, Services get populated Endpoints, PVCs bind, and Jobs complete — all immediately and deterministically (no controller goroutines). Core, apps, batch, networking, rbac, storage, autoscaling, discovery, and policy groups are served, with `/scale` and `/status` subresources, label/field selectors, and `?watch=true` streaming (selector-filtered) so real `Informer` / `Reflector` machinery works. Data-plane lists are unpaginated (`limit`/`continue` are ignored — every list returns the full set).

Non-goals are deliberate emulation boundaries: no kubelet-backed Pod subresources (`/log`, `/exec`, `/attach`, `/portforward`), no real scheduling (single synthetic node, no affinity/taints), no admission/quota/policy **enforcement** (ResourceQuota, LimitRange, NetworkPolicy, RBAC, PDB are stored but not enforced), HPA does not autoscale, CronJob does not fire on schedule, and rollouts converge instantly with no surge pacing or revision history. See `docs/services.md` §18 for the full resource list.
The data plane now covers CustomResourceDefinitions (dynamic servable kinds), server-side apply with `managedFields` field ownership + conflict detection, `?dryRun=All`, finalizer-gated deletion, `?limit=&continue=` pagination, synthetic `pods/log` + PDB-gated `pods/eviction`, `metrics.k8s.io` (`kubectl top`) + HPA actuation, object-count ResourceQuota / LimitRange / PDB enforcement, RBAC SubjectAccessReview + NetworkPolicy evaluation, opt-in admission webhooks, watch `resourceVersion` resume + BOOKMARK, and a deterministic injectable clock. Remaining emulation boundaries are deliberate simplifications: no real kubelet (synthetic logs; `exec`/`attach`/`portforward` return a typed 501), no scheduling beyond the single synthetic node (DaemonSet `nodeSelector` honored; no affinity/taints), admission webhooks call out only when explicitly enabled, RBAC/NetworkPolicy are queryable rather than request-time-enforced, CronJob fires via `TickCronJobs` (no wall clock), and rollouts converge instantly. See `docs/services.md` §18 for the full resource list.

Two provider-specific services also ship as full SDK-compat handlers. **AWS Bedrock** covers the `bedrock` control plane (foundation models, customization jobs, custom models, guardrails with policy configs + versions, provisioned throughput, invocation logging, resource tagging, model import/copy/evaluation jobs, inference profiles, prompt routers, marketplace model endpoints, foundation-model agreements, and automated-reasoning policies) and the `bedrock-runtime` data plane (InvokeModel with family-aware response envelopes, Converse, streaming ConverseStream / InvokeModelWithResponseStream over `vnd.amazon.eventstream`, CountTokens, ApplyGuardrail, and async invoke). A companion **AWS Bedrock Agent** handler covers the `bedrock-agent` control plane (agents, knowledge bases, data sources, flows, prompts) and the `bedrock-agent-runtime` data plane (InvokeAgent streaming, Retrieve, RetrieveAndGenerate); its runtime handler registers before the control plane and matches only POST so the two never collide on the shared `/agents` and `/knowledgebases` roots. `bedrock-agent` coverage is intentionally scoped to this core resource lifecycle and runtime data plane — agent versioning/aliases beyond basic create, action groups, and agent collaborators are out of scope for this iteration. **Azure Databricks** covers the `armdatabricks` ARM workspace resource plus the `databricks-sdk-go` workspace data plane — clusters, instance pools, jobs and runs, cluster policies, libraries, permissions, secrets, tokens, git credentials, repos, DBFS, workspace notebooks/directories, SQL warehouses, pipelines, serving endpoints, SCIM identity, and Unity Catalog.

Expand Down
30 changes: 24 additions & 6 deletions docs/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -1668,19 +1668,37 @@ It behaves like a tiny always-converged cluster (minikube-like) rather than a ba

**Core (`core/v1`)**: Namespace, ConfigMap, Secret (StringData merged into Data), ServiceAccount (`default` auto-created per namespace), Pod (driven **Running** with a synthetic Pod IP — a directly-created Pod with a terminal phase is preserved), Service (ClusterIP from 10.96.0.0/12, immutable on update), Endpoints (get/list/watch only — auto-managed per Service), PersistentVolumeClaim (→ Bound), PersistentVolume (→ Available), Node, Event, ResourceQuota, LimitRange.

**Workload controllers (`apps/v1`)**: Deployment, ReplicaSet, StatefulSet (stable `-0..-N` names + one Bound PVC per `volumeClaimTemplate`), DaemonSet (one Pod per node). All materialize Running Pods owned via `ownerReferences`; deleting a controller cascade-deletes its Pods and drains Endpoints. A change to the pod template (e.g. image) is a **rolling update** — stale-hash Pods (tracked by a `pod-template-hash` label) are replaced. Deployments/StatefulSets expose **`/scale`** and **`/status`** subresources.
**Workload controllers (`apps/v1`)**: Deployment, ReplicaSet, StatefulSet (stable `-0..-N` names + one Bound PVC per `volumeClaimTemplate`), DaemonSet (one Pod per node whose labels satisfy the template `nodeSelector` — zero Pods when it doesn't match the synthetic node). A **Deployment interposes a ReplicaSet** per pod-template revision (Deployment→RS→Pod, matching real topology), and a template change creates a new ReplicaSet and deletes the old one outright — an instantaneous swap (no `revisionHistoryLimit`, no `kubectl rollout undo`, no surge/unavailable pacing). All materialize Running Pods owned via `ownerReferences`; deleting a controller cascade-deletes the chain and drains Endpoints. Deployments/StatefulSets expose **`/scale`** and **`/status`** subresources. CronJob scheduling is driven explicitly via `TickCronJobs()` (no background timer), which performs real due-evaluation against the cluster clock: it parses the standard 5-field `spec.schedule` (`*`, `*/n`, lists, `a-b` ranges) and materializes a Job only when a scheduled time falls in `(status.lastScheduleTime, now]` — advancing `lastScheduleTime` to the fired slot so re-ticking the same instant never double-creates — and honors `concurrencyPolicy` (`Forbid`/`Replace`/`Allow`) and `startingDeadlineSeconds`.

**Other groups** (registry-backed CRUD + list/watch/patch/delete): `batch/v1` Job (→ Succeeded Pods) / CronJob; `networking.k8s.io/v1` Ingress (→ load-balancer IP) / IngressClass / NetworkPolicy; `rbac.authorization.k8s.io/v1` Role / RoleBinding / ClusterRole / ClusterRoleBinding; `storage.k8s.io/v1` StorageClass; `autoscaling/v2` HorizontalPodAutoscaler; `discovery.k8s.io/v1` EndpointSlice; `policy/v1` PodDisruptionBudget.
**Other groups** (registry-backed CRUD + list/watch/patch/delete): `batch/v1` Job (→ Succeeded Pods) / CronJob; `networking.k8s.io/v1` Ingress (→ load-balancer IP) / IngressClass / NetworkPolicy; `rbac.authorization.k8s.io/v1` Role / RoleBinding / ClusterRole / ClusterRoleBinding; `storage.k8s.io/v1` StorageClass; `autoscaling/v2` HorizontalPodAutoscaler; `discovery.k8s.io/v1` EndpointSlice; `policy/v1` PodDisruptionBudget; `apiextensions.k8s.io/v1` CustomResourceDefinition; `admissionregistration.k8s.io/v1` Mutating/ValidatingWebhookConfiguration.

**Selectors**: label selectors on list, and field selectors for the fields the store can answer (`metadata.name`, `metadata.namespace`, Pod `status.phase` / `spec.nodeName`). List responses are unpaginated — `limit`/`continue` are not honored (an emulation simplification; every list returns the full set in one response).
**Custom resources (CRDs)**: creating a `CustomResourceDefinition` dynamically materializes a servable store for every served version — the custom-resource kind is then served by the generic handler (CRUD/list/watch/`/status`) and advertised in discovery immediately; the CRD is marked `Established`. Deleting the CRD deregisters the kind and cascade-deletes its custom resources — including when the CRD carries a finalizer, in which case teardown runs once the last finalizer drains. Structural schema validation of CRs is a documented simplification (accept-and-store).

**Patch**: all four content types — JSON-merge-patch, JSONPatch (RFC 6902), and strategic-merge-patch (real strategic merge against the typed struct for core/apps kinds, so `kubectl set image` merges the container list by name). Server-side-apply is accepted and applied as a merge (an emulation simplification — apply field-ownership is not tracked).
**Selectors & pagination**: label selectors on list; field selectors for `metadata.name` / `metadata.namespace`, Pod `status.phase` / `spec.nodeName`, and Event fields (`involvedObject.name/namespace/kind/uid`, `reason`, `type`). List responses honor **`?limit=&continue=`** chunked pagination across the registry and typed list paths: the `metadata.continue` token is key-anchored (it encodes the last object's `namespace/name`), so an insert or delete before that key cannot skip or duplicate later items under concurrent mutation, and a malformed token returns `410 Gone` (reason `Expired`) per client-go's pager contract. A well-formed token whose key was since deleted resumes gracefully at the next greater key rather than `410`-ing on a compacted resourceVersion — strictly more forgiving than upstream.

**Patch & server-side apply**: JSON-merge-patch, JSONPatch (RFC 6902), and strategic-merge-patch (real strategic merge against the typed struct for core/apps kinds, so `kubectl set image` merges the container list by name). **Server-side apply** (`application/apply-patch+yaml`) tracks per-`fieldManager` field ownership in `metadata.managedFields`; an apply that changes a field owned by another manager returns **409 Conflict** unless `?force=true` (which transfers ownership), and an owner re-applying the same value is a no-op. A re-apply by the same manager that omits a field it previously owned removes that field, unless another manager also owns it. Plain PUT/PATCH updates record an `Update`-operation `managedFields` entry for their `fieldManager` (defaulted from the User-Agent when absent), taking or sharing ownership rather than conflicting (only Apply-vs-Apply is a 409). Ownership is tracked at leaf granularity (map keys / whole arrays) — per-element list merging is not modeled, a documented subset of upstream SSA.

**Dry-run**: writes with `?dryRun=All` (`kubectl apply|create|delete --dry-run=server`) run validation, defaulting, and quota admission (a create against an at-limit namespace returns the same `403` a real create would), echo the object the server would store, and persist nothing — no resourceVersion bump, reconcile, quota reservation, or watch event.

**Finalizers**: an object carrying `metadata.finalizers` goes **Terminating** on delete (`deletionTimestamp` stamped, object retained) and is removed only when the last finalizer is cleared via update/patch — on the registry path and typed Namespace/Pod. Finalizers are also honored during cascade: a finalizer-bearing child reached by owner garbage-collection or namespace teardown goes Terminating rather than being reaped, until its finalizers drain. The server-owned `deletionTimestamp` survives a merge-patch — an RFC-7396 `null` cannot resurrect a Terminating object.

**Pod subresources**: `pods/{name}/log` returns synthetic container output; `exec`/`attach`/`portforward` return a typed `501` (they need a streaming protocol upgrade the emulator doesn't implement); `pods/{name}/eviction` honors PodDisruptionBudgets.

**Metrics & autoscaling**: `metrics.k8s.io/v1beta1` (`kubectl top`) serves synthetic Pod/Node metrics from the live pods + synthetic node; a HorizontalPodAutoscaler reconcile drives its target Deployment on a Resource CPU `averageUtilization` metric — sampling the target Pods' CPU from that metrics source and applying the real HPA ratio `desiredReplicas = ceil(currentReplicas × currentUtilization ÷ targetUtilization)`, clamped into `[minReplicas, maxReplicas]` — and falls back to a plain min/max clamp when no CPU metric is configured or the target Pods declare no CPU request, reporting `currentReplicas`/`desiredReplicas`/`currentMetrics` on status.

**Policy enforcement**: object-count **ResourceQuota** is enforced on create (403 over limit) and on server-side dry-run; `status.used` is updated on create and recomputed from the live count on delete (it tracks the live object count rather than climbing monotonically); **LimitRange** applies container defaults and min/max validation on pod create; **PodDisruptionBudget** gates `pods/eviction` (429 when eviction would violate the budget); **RBAC** is queryable via `authorization.k8s.io/v1` SubjectAccessReview (evaluated against stored Roles/ClusterRoles + bindings); **NetworkPolicy** is queryable via an in-process evaluation (no live traffic).

**Admission webhooks** (opt-in): Mutating/ValidatingWebhookConfiguration objects store and round-trip through `kubectl apply`. With admission explicitly enabled (`APIServer.SetAdmissionEnabled`), create/update/patch calls matching webhooks apply mutations and honor denials (4xx); it is off by default so the data plane stays zero-network and deterministic.

**Watch resume**: a watch with `resourceVersion>0` skips the initial snapshot replay and streams only subsequent events; `allowWatchBookmarks=true` emits a post-sync BOOKMARK carrying the current resourceVersion. A slow watcher that overflows its buffer gets a `410 Gone` so `client-go` relists.

**Deterministic time**: every data-plane timestamp (creationTimestamp, pod start/conditions, managedFields) is sourced from an injectable clock (`APIServer.SetClock`); a `config.FakeClock` makes them fully deterministic for tests.

**Watch streaming**: each list endpoint accepts `?watch=true` and upgrades to a `Transfer-Encoding: chunked` JSON event stream (`{"type":"ADDED|MODIFIED|DELETED","object":{...}}`). Initial state replays as ADDED events on subscribe, and the request's `labelSelector`/`fieldSelector` filters both the initial snapshot and live events, so `client-go` `Informer` / `SharedIndexInformer` machinery (operator-sdk, Helm, ArgoCD, …) — including selective informers — just works. A fresh cluster bootstraps a synthetic Ready node (`cloudemu-node-0`), and each selector Service's endpoints are mirrored into a `discovery.k8s.io` **EndpointSlice** so EndpointSlice-mode consumers see the same backends as the `Endpoints` object.

**Cascade**: deleting a Namespace or an owning controller publishes DELETED events for every child resource (garbage collection follows `ownerReferences`).
**Cascade**: deleting a Namespace or an owning controller publishes DELETED events for every child resource (garbage collection follows `ownerReferences`) — finalizer-bearing children instead go Terminating (MODIFIED) until drained.

**Non-goals** (deliberate emulation boundaries): no kubelet-backed Pod subresources (`/log`, `/exec`, `/attach`, `/portforward`); no real scheduling (all Pods land on a single synthetic node, no affinity/taints/resource-fit); no admission/quota/policy **enforcement** (ResourceQuota, LimitRange, NetworkPolicy, PodDisruptionBudget, RBAC are stored and served but not enforced); HPA does not actually autoscale; CronJob does not fire on a schedule; rollouts converge instantly with no surge/unavailable pacing or revision history; no aggregated API servers, admission webhooks, or CRD registration.
**Emulation boundaries** (deliberate simplifications, not gaps): there is no real kubelet — Pods are driven Running synthetically and `pods/log` is synthetic while `exec`/`attach`/`portforward` return a typed 501; no real scheduling beyond the single synthetic node (DaemonSet `nodeSelector` is honored, but affinity/taints/resource-fit are not); admission webhooks make outbound calls only when explicitly enabled (off by default to stay zero-network); server-side apply tracks ownership at leaf granularity (no per-element list merge); NetworkPolicy and RBAC are **queryable** (SubjectAccessReview / EvaluateNetworkPolicy) rather than request-time-enforced, since the emulator has no packet path or authenticated identity; CronJob has no wall-clock timer (schedules are evaluated only when `TickCronJobs` is called) and supports only the standard 5-field cron syntax (nonstandard `@`-macros, `L`/`W`/`#`/`?` characters, and seconds/year fields are rejected); rollouts converge instantly (no surge/unavailable pacing, minimal revision history); and OpenAPI is served cluster-independently, so CRD schemas aren't published there (custom resources still work via discovery).

---

Expand Down
Loading
Loading