Add MIT license - #8
Merged
Merged
Conversation
6 tasks
NitinKumar004
added a commit
that referenced
this pull request
Jul 30, 2026
…ocs) From thzgajendra's review of PR #299: - Lint (#1): replace the version-sensitive //nolint:prealloc directives with real preallocation (coreResources/appsResources/registryAPIResources, openapi kinds, endpoint-address slice), so golangci-lint is clean regardless of prealloc's version-dependent placement — no dangling directives. - Docs (#2): drop the 'pagination' claim from the k8s data-plane sentence in sdk-server.md; data-plane lists are unpaginated, matching services.md. - Watch load-shedding (#3): a slow watcher that overflows its buffer now receives a 410 Gone (ERROR) event and the stream ends, so client-go relists instead of running with a permanently-divergent cache. Regression test added. - GC comment (#4): correct the 'never mutate while ranging' wording — deleting the current key mid-range is legal; the BFS is what makes the cascade order-independent. - Provider package headers (#8): describe the now-wired data plane instead of 'out of scope / Wave 2'. Deferred as documented follow-ups (all Low): clamp/silent-signal, Job shrink reconcile, committed kubectl smoke test, provider CA-vs-sentinel misconfig window.
NitinKumar004
added a commit
that referenced
this pull request
Jul 30, 2026
…upport (#299) * feat(k8s): connect parity across EKS/AKS/GKE via a shared CA (Phase A) The data-plane serving cert and every provider's advertised CA must be the same authority or client-go's TLS handshake fails. Extract the CA into a new internal/k8spki package used by both the serving TLS config and all three control planes: - EKS: tls.go now delegates to k8spki (serve + EKS call sites unchanged). - GKE: advertise the real CA in masterAuth.clusterCaCertificate; drop the unparseable dummy blob that broke the handshake outright. - AKS: embed the real CA in the rendered kubeconfig and drop insecure-skip-tls-verify — parity with EKS/GKE. Tests: AKS data-plane test now serves with the k8spki cert and validates end-to-end (no skip-verify); new GKE real-TLS connect-parity test proves the advertised CA certifies the endpoint (create cluster -> validate CA -> client-go ConfigMap round-trip). RenderKubeconfig test updated to the real-CA behavior. First phase of the k8s runtime/parity work; registry refactor + reconcile engine + workload kinds follow on this branch. * feat(k8s): parse resource subresources in the router (Phase B foundation) Add Route.Subresource and parse the /{name}/{subresource} tail for both cluster-scoped (/api/v1/nodes/n/status) and namespaced (/apis/apps/v1/namespaces/ns/deployments/d/scale) shapes. ServeHTTP routes subresource requests to a dedicated dispatcher (stubbed to 404 until the reconcile phase wires /status and /scale) so a subresource path is never mis-parsed as a write against the parent object. Updated the parseRoute unit test to the new (correct) cluster-subresource semantics. * feat(k8s): generic resource registry + reconcile engine (Phases B+C) Turn the k8s data plane from a CRUD store into a minikube-like runtime. Registry (registry.go, registry_ops.go, registry_defs.go): a generic unstructured-backed store + one handler serving CRUD, list (label & field selectors), watch, patch, delete (ownerReference garbage collection), and the /status + /scale subresources for any registered kind. New kinds are a registration + optional reconcile hook. Registers apps/v1 ReplicaSet, StatefulSet, DaemonSet and core/v1 PersistentVolumeClaim; discovery is derived from the registry so it can't drift. Reconcile engine (reconcile.go), run synchronously on every write (no controller goroutines, so it stays deterministic): - Pods are driven Running with a synthetic Pod IP and ready containers. - Deployment materializes its Pods and reports real status; ReplicaSet and DaemonSet do likewise; StatefulSet creates stable-ordinal Pods (name-0..N-1) plus a Bound PVC per volumeClaimTemplate. - The endpoints controller fills a Service's Endpoints from the Running Pods matching its selector, and drains them when Pods are deleted/GC'd. - Deleting a controller cascades to its Pods; scaling (spec.replicas or the /scale subresource) adjusts the Pod count. Typed handlers: Deployment now reconciles + serves /scale and /status; direct Pod creates come up Running; Pod list honors label/field selectors; Service create populates endpoints. Tests: new client-go WorkloadRuntime E2E (Deployment+Service -> Running pods + endpoints -> scale to 4 -> StatefulSet with 3 stable pods + 3 Bound PVCs -> DaemonSet -> cascade teardown). Existing pod/cascade/provider tests updated to the new Running/materialized behavior. Deployment /scale + /status advertised in discovery. Deferred to later phases: the intermediate ReplicaSet object for Deployments (pods are owned by the Deployment directly); Job/CronJob, Ingress, RBAC, HPA, Node, Event, NetworkPolicy, EndpointSlice; strategic-merge / server-side-apply. * k8s: register batch/networking/rbac/storage/autoscaling/discovery + core supporting kinds; registry-driven discovery Adds registry entries for Job/CronJob, Ingress/IngressClass/NetworkPolicy, RBAC (Role/RoleBinding/ClusterRole/ClusterRoleBinding), StorageClass, HorizontalPodAutoscaler, EndpointSlice, and core PVC/PV/Node/Event/ ResourceQuota/LimitRange. Reconcile hooks drive Job pods to Succeeded, Ingress to a load-balancer IP, and PV to Available/Bound. serveDiscovery now derives the /apis group list and every /apis/<group>/<version> resource list from registeredResources() (seeded with the typed apps/policy groups) instead of a hardcoded switch, so new groups and their subresources are discoverable by kubectl and client-go without drifting from what the server serves. * test(k8s): e2e coverage for supporting kinds via client-go Drives Jobs (complete to Succeeded with materialized pods), Ingresses (get a load-balancer IP), PVCs (bind), StorageClass/RBAC/HPA/Node round-trips, and asserts the new API groups are discoverable — the negotiation kubectl and client-go do before any typed request. * k8s: rolling updates replace Pods on pod-template change; advertise endpoints - Controllers stamp a pod-template-hash label on Pods; reconcile treats a changed template hash as a rolling update and replaces stale-hash Pods (Deployment/ReplicaSet via syncScaledPods, StatefulSet via syncStablePods). Convergence is instant — no surge/unavailable pacing. - buildControllerPod now copies the template label map before stamping, so it can't mutate the controller's shared template. - Discovery advertises core/v1 endpoints (get/list/watch only, matching the read-only handler) so kubectl/client-go can resolve them. E2E: runtime test now exercises a rolling update (image change replaces all Pods, endpoints re-point); supporting-kinds test asserts endpoints discovery. * docs(k8s): document the minikube-like data plane and its non-goals Update services.md §18, sdk-server.md, and the package doc to reflect the reconcile engine (Running Pods, Endpoints, binding PVCs, completing Jobs), validated TLS via the shared CA, the full multi-group resource surface with /scale and /status subresources, rolling updates, and the deliberate emulation boundaries (no exec/logs/portforward, no scheduling, no quota/RBAC/policy enforcement, no HPA/CronJob actuation). * fix(k8s): merge-patch to /scale no longer silently scales to zero applyUnstructuredPatch decoded the merged JSON with plain json.Unmarshal into map[string]any, which turns whole-number JSON into float64. unstructured.NestedInt64 accepts only int64, so spec.replicas read back as 0 — a 'kubectl scale --replicas=N' (a merge-patch to the /scale subresource, or any merge-patch touching replicas) silently scaled the workload to zero. Decode via unstructured.Unstructured.UnmarshalJSON instead, which preserves integers as int64. This fixes every merge-patch path (object and /scale) at the root. Regression test drives a merge-patch scale-up and asserts both the returned Scale and the stored object carry replicas=4. * fix(k8s): address review findings across reconcile, GC, endpoints, PKI - Endpoints: only bump ResourceVersion / publish MODIFIED when the address set actually changes. resyncEndpointsForNamespaceLocked runs for every Service on any Pod change, so an unchanged Service was emitting a spurious watch event (with a climbing RV) on unrelated Pod churn. Regression test added. - Pod field selector: support spec.nodeName. Every materialized Pod is scheduled to the synthetic node, so 'kubectl get pods --field-selector spec.nodeName=...' (node-drain/kubelet tooling) previously returned an empty list. E2E covers it. - Garbage collection: walk the owned set breadth-first and collect UIDs before deleting, instead of mutating each store's map while ranging it and recursing. Pods owned by an intermediate controller (not just the root) are now reaped. - Scale subresource: only bump generation when spec.replicas actually changes, matching registryUpdate/registryPatch (no spurious generation != observed). - Job: ignore a non-positive spec.completions (default to 1) so a Job can't report Complete having run zero Pods. - StatefulSet PVCs: deep-copy the volumeClaimTemplate spec per ordinal instead of aliasing one map across every PVC. - PKI: give each serving leaf a random 128-bit serial (was fixed '2') and assert BasicConstraintsValid (cA=FALSE) so strict non-Go verifiers accept the leaf. - docs: correct the data-plane list to say it is unpaginated (limit/continue are not honored) and scope field-selector support accurately. * k8s: real-user kubectl parity — protobuf writes, OpenAPI v2/v3, strategic & JSON patch Driving a running cloudemu server with real kubectl (cluster created via the EKS/GKE/AKS SDK, then kubectl against the advertised endpoint) surfaced gaps the JSON-forcing client-go tests masked. kubectl now works end-to-end. - Protobuf request bodies: kubectl sends built-in kinds as protobuf on writes and does NOT retry as JSON on 415, so every 'kubectl create/apply/scale' write failed. Decode protobuf via the client-go scheme's recognizing deserializer (typed handlers decode in place; registry handlers convert to unstructured). Responses stay JSON — clients' Accept allows it. - OpenAPI: serve a v3 discovery root + per-group docs that carry each served GVK (with a permissive schema) so kubectl resolves the kind and stays on the JSON v3 path; and serve the legacy v2 doc as protobuf bytes (mime-safe application/octet-stream content type) for the fallback. Without this 'kubectl apply' died at 'failed to download openapi'. Served cluster-independently in APIServer.ServeHTTP so the prefix-less v3 serverRelativeURL follow-ups resolve. - Patch types: typed handlers now accept strategic-merge-patch (kubectl's default for set/edit/label — real strategic merge, so the container list merges by name) and JSONPatch (RFC 6902), in addition to merge-patch; registry handlers gain JSONPatch too. - Discovery: advertise kubectl short names (pvc, hpa, sts, ds, rs, ing, sc, …) for registry kinds so 'kubectl get pvc' resolves. Verified with real kubectl v1.36 against a standalone server: full lifecycle (apply → scale → rolling update → statefulset/PVCs → daemonset → job → cronjob → ingress → hpa → pv/pvc/storageclass → rbac → networkpolicy → node → all three patch types → cascade teardown) across EKS, GKE, and AKS connect paths. * docs(k8s): note full kubectl parity (protobuf, OpenAPI, all patch types) * k8s: drop len()+1 map capacity hint (clears CodeQL allocation-overflow alert) * k8s: filter watch streams by label/field selector (review blocker) Watch streams ignored labelSelector/fieldSelector: typed watches (watchPods, watchDeployments, …) filtered neither initial nor streamed events, and the registry watch filtered only the initial snapshot. A selective watch ('kubectl get pods -l app=x -w', or any informer/controller-runtime cache built with a selector) therefore received non-matching objects — polluting reflector caches and firing spurious reconciles, which the reconcile engine amplifies (one Deployment emits many Pod events). streamWatch now takes a keep(T) predicate applied to both the initial snapshot and every streamed event; each watch handler builds it from the request's selectors (parseListSelectors + metaFieldsMatch/podMatchesFields/matchesFields). Also extracts field-selector-name constants (fixes goconst) and indexes filterPods to avoid per-item Pod copies. * k8s: cap materialized pods, bootstrap Node, mirror EndpointSlices, resync endpoints on pod update/patch Correctness gaps from the PR review: - Unbounded replicas/completions: the reconciler runs synchronously under the cluster lock, so a huge spec value would allocate/hang the whole API. Clamp the materialized Pod count to maxReconciledPods (500) for Deployment, ReplicaSet/StatefulSet (via replicasOf), and Job. reconcileJob's top-up is also made O(n) instead of O(n²). - Synthetic Node: bootstrap cloudemu-node-0 (Ready, InternalIP) in newClusterState so 'kubectl get nodes' is non-empty and the node every Pod is scheduled onto actually exists. - EndpointSlices: mirror each Service's endpoints into a discovery.k8s.io EndpointSlice (labelled kubernetes.io/service-name) so EndpointSlice-mode consumers (kube-proxy, Gateway API) see the same backends as Endpoints. - Pod update/patch now resync endpoints (a label change matching a Service selector was invisible until unrelated churn) and re-drive a spec-only PUT back to Running so it isn't dropped out of the endpoint set. reconcileServiceEndpointsLocked is split into matchingEndpointAddressesLocked / writeEndpointsLocked / syncEndpointSliceLocked (also lowers its complexity). E2E asserts the synthetic Node and populated EndpointSlices; a new watch test asserts label-selector stream filtering. * k8s: dedup watch handlers + resolve golangci-lint to zero (review blocker) - All 8 typed watch handlers now share a generic serveWatch[T] helper (removes the dupl the near-identical subscribe/snapshot/stream blocks triggered, and addresses the reuse the review flagged). - golangci-lint (repo .golangci.yml, v2.11.4) is clean on the new non-test files: extracted goconst constants (api/apis path segments, status/scale subresources, group names), named crypto/mnd magic numbers in k8spki, gave parseListSelectors named results, indexed the container-status range, lowered ServeHTTP/serveRegistry complexity via dispatchResource / serveRegistryItem, fixed the govet shadow, and added reasoned //nolint for the legitimately-global lookup tables and hugeParam k8s structs. No behavior change; build, vet, tests, and -race remain green. * k8s: bump Deployment generation on spec change; unify /scale patch types - Typed Deployment now sets metadata.generation=1 on create and advances it only on a spec change (update/patch), matching apiserver semantics and the registry path so observedGeneration comparisons are meaningful. - deploymentScale PATCH routes through the shared applyPatchBytes dispatcher, so the typed /scale honors merge / strategic-merge / JSONPatch like the registry /scale (the two paths no longer diverge). - Tidy a stale AKS kubeconfig comment (Phase 3 -> the normal path). * docs(k8s): note watch selector filtering, synthetic node, EndpointSlice mirroring * k8s: address second-review findings (lint robustness, watch relist, docs) From thzgajendra's review of PR #299: - Lint (#1): replace the version-sensitive //nolint:prealloc directives with real preallocation (coreResources/appsResources/registryAPIResources, openapi kinds, endpoint-address slice), so golangci-lint is clean regardless of prealloc's version-dependent placement — no dangling directives. - Docs (#2): drop the 'pagination' claim from the k8s data-plane sentence in sdk-server.md; data-plane lists are unpaginated, matching services.md. - Watch load-shedding (#3): a slow watcher that overflows its buffer now receives a 410 Gone (ERROR) event and the stream ends, so client-go relists instead of running with a permanently-divergent cache. Regression test added. - GC comment (#4): correct the 'never mutate while ranging' wording — deleting the current key mid-range is legal; the BFS is what makes the cascade order-independent. - Provider package headers (#8): describe the now-wired data plane instead of 'out of scope / Wave 2'. Deferred as documented follow-ups (all Low): clamp/silent-signal, Job shrink reconcile, committed kubectl smoke test, provider CA-vs-sentinel misconfig window.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan