Skip to content

OSC Platform Networking and Isolation Model

Jump to bottom
Jonas Birmé edited this page Aug 27, 2026 · 2 revisions

OSC Platform Networking and Tenant Isolation Model

Verified against source and live infrastructure: infra-osaas commit 275182d3623481f734f52b05d370f217235a9c4b, osaas-deploy-manager commit c06f6af00f3d002eaf40f4c02cb5ffe2735994bf, and osaas-lib-orchestrator commit 32b9811db2234adea470085493992f20f65d2654, plus direct read-only verification against both production Kubernetes clusters, all as of 2026-08-27. Every claim below is sourced to a specific file and line in one of these repositories, confirmed live against the running clusters, or marked TBD, unverified where it could not be confirmed either way.

This page describes how network traffic reaches a service instance or My App on Eyevinn Open Source Cloud, and what is and is not known about isolation between tenants sharing the platform's Kubernetes clusters. It exists to answer security and architecture questions accurately, including for prospective customers evaluating OSC. Where the underlying evidence does not settle a question, that is stated explicitly rather than assumed.

How a service instance or My App becomes reachable

Every catalog service, including the My Apps runners (Web Runner, Python Runner, .NET Runner, Go Runner, WASM Runner), is deployed through the same per-service orchestrator library, osaas-lib-orchestrator. When an instance is created, the library provisions a Kubernetes Deployment, a ClusterIP (or NodePort, for services with allocated TCP ports) Service, and, for the first instance of that service in an environment, a shared Ingress object named {serviceName}-osaas-{environment} (osaas-lib-orchestrator/src/instance_service/k8s.ts:2030).

That shared ingress carries the annotations that make the platform's auth model work:

  • nginx.ingress.kubernetes.io/auth-url pointing at https://api-{apiDomain}/authenticate, the auth gate that checks the caller's session or Service Access Token before nginx proxies the request onward (osaas-lib-orchestrator/src/instance_service/k8s.ts:2033).
  • A server-snippet that redirects browser navigations to the OSC login gate on a 401, and returns a 401 for everything else (osaas-lib-orchestrator/src/instance_service/k8s.ts:2037).
  • cert-manager.io/issuer, with acme-challenge-type: dns01 and acme-dns01-provider: route53, so TLS certificates for each service's public hostnames are issued and renewed automatically via cert-manager (osaas-lib-orchestrator/src/instance_service/k8s.ts:2066-2068).

The public hostname for an instance is generated as {appName}.{apiDomain}, where apiDomain defaults to {serviceTypeId}.{environment}.osaas.io but is overridden per service via the API_DOMAIN environment variable (osaas-lib-orchestrator/src/utils/k8s.ts:17-24). In practice, services set this to the *.auto.{environment}.osaas.io pattern, for example encore/prod-se.yml:7 sets api_domain: "encore.auto.prod-se.osaas.io" in infra-osaas. This is the pattern behind the service-instance URLs you see in the web console (*.auto.prod.osaas.io and, since the Elastx cutover, *.auto.prod-se.osaas.io). It is a different mechanism from a My App's default *.apps.osaas.io URL or a custom domain; those are handled by a separate proxy-ingress path in osaas-deploy-manager and are out of scope for this page.

Ingress traffic for the platform is handled by ingress-nginx (infra-osaas/ingress-nginx.yml:385-395, controller version 1.13.9 at the cited commit). My Apps traffic has its own dedicated ingress controller, IngressClass osaas-myapps-nginx (infra-osaas/osaas-custom-domain/ingress-controller.yml), but this page's line-level citations focus on the shared per-service ingress path described above since that is what could be directly verified.

Kubernetes namespace model: one namespace per service, not per tenant

This is the load-bearing fact for everything below. Every tenant's instance of a given service lands in the same Kubernetes namespace, named after the service (for example, all customers' Valkey instances run in the valkey-io-valkey namespace). Namespaces are not created per tenant and are not created per instance.

This is stated directly in the orchestrator library's own source comment:

"The instance namespace is shared by every instance of the service and is never torn down." (osaas-lib-orchestrator/src/instance_service/k8s.ts:1414)

Supporting evidence for the same conclusion:

  • createNamespacedDeployment for a tenant's instance is called with namespace: this.namespace, where this.namespace is a constructor argument set once per orchestrator process (i.e., once per service), not derived per tenant or per instance (osaas-lib-orchestrator/src/instance_service/k8s.ts:1891-1894, constructor at osaas-lib-orchestrator/src/instance_service/k8s.ts:131-145).
  • The per-instance Service object is likewise created with namespace: this.namespace and a selector scoped only by an instance label, not by any namespace-level tenant boundary (osaas-lib-orchestrator/src/instance_service/k8s.ts:1327-1332).
  • Instance and app names are generated as {customer}-{instanceName} via generateSafeName (osaas-lib-orchestrator/src/utils/k8s.ts:13-15), which is deterministic and derivable from information a tenant already knows about themselves (their own tenant ID and instance name), not a random or secret identifier.

Cross-tenant network reachability

This is the highest-stakes claim on this page. It has been verified both against source and live against both production clusters — read this section carefully rather than the summary alone.

What is confirmed: No NetworkPolicy object exists anywhere in infra-osaas. A repository-wide search (grep -rl "kind: NetworkPolicy" .) returns zero matches at the cited commit. Kubernetes NetworkPolicy is the standard mechanism for restricting pod-to-pod traffic within or across namespaces; none is defined for any OSC-managed namespace in this repository. Combined with the "one namespace per service, shared across every tenant" model above, and the fact that per-instance Service objects default to ClusterIP (reachable from any pod in the cluster via Kubernetes' built-in Service DNS, unless something restricts it), the Kubernetes objects that OSC itself defines do not implement pod-to-pod network segmentation between tenants sharing the same service's namespace, or between tenants' pods and pods in other namespaces on the same cluster.

Concretely: the auth-url / auth-gate mechanism described above protects the external HTTPS ingress path only. It is an nginx auth_request check performed by the ingress controller before proxying a request from outside the cluster to a backend Service. It has no bearing on traffic that never passes through that ingress, such as one pod calling another pod's ClusterIP Service directly over the cluster's internal pod network.

Verified directly against both production clusters: no network-policy enforcement currently restricts pod-to-pod traffic between tenant namespaces. This was confirmed live, not inferred from the absence of manifests alone — the underlying cluster network layer on both Linode LKE and Elastx prod-se was checked directly, and neither applies any tenant-segmentation policy independent of the (absent) NetworkPolicy objects described above.

Also verified directly: tenant workload pods do not have a Kubernetes RBAC path to enumerate the cluster. No explicit serviceAccountName is set on tenant containers in osaas-lib-orchestrator (the serviceAccountName: 'api' seen in the codebase belongs to the per-service orchestrator's own control-plane deployment, not to tenant workload pods), so tenant pods run under their namespace's default ServiceAccount — and that ServiceAccount was confirmed, live on both production clusters, to have no permission to list or get pods, namespaces, or secrets outside its own namespace, or even to list secrets within its own namespace. A tenant's own application code cannot use the Kubernetes API to discover other tenants' resources.

Practical summary: a tenant's application code, if it could execute arbitrary outbound network calls, would need to know or guess another tenant's Service DNS name to attempt direct access — a bar that varies by tenant, since some tenant IDs are effectively public (derived from the customer's own company or product name, visible in support tickets, custom domains, or marketing materials) while others are not. Given the confirmed absence of both network-level segmentation and any Kubernetes-API path to discover other tenants automatically, this reduces to whether that direct-access attempt, if the DNS name were known or guessed, would succeed against a ClusterIP Service on the shared pod network — which it would, absent any restriction found on either cluster. This is a confirmed platform characteristic today, not a theoretical one.

Resource isolation between tenants

CPU and memory limits

No resources.limits or resources.requests for CPU or memory were found on the main application container in the Deployment spec that osaas-lib-orchestrator builds for a service instance (osaas-lib-orchestrator/src/instance_service/containers.ts, and confirmed by a repository-wide search for resources across osaas-lib-orchestrator/src/instance_service/k8s.ts, which returns only two matches, both PersistentVolumeClaim storage requests at lines 1146 and 1211, not container CPU/memory). The only container-level resources found anywhere in this code path is a 128Mi memory request on an init container used when a persistent volume is attached (osaas-lib-orchestrator/src/instance_service/containers.ts:329-333); this is not a limit, and it does not apply to the main application container.

Because My Apps runners are deployed through this same orchestrator mechanism, this applies to My Apps as well as catalog service instances.

TBD, unverified: Whether CPU/memory limits are enforced by some mechanism outside osaas-lib-orchestrator and osaas-deploy-manager (for example, a Kubernetes LimitRange or ResourceQuota applied directly to a service's namespace, outside of this orchestrator code). A repository-wide search of infra-osaas for kind: LimitRange found exactly three matches, none of which are a general policy for tenant service or My App namespaces: encore/mem-limit-range (namespace encore, default container memory limit 15Gi, default request 512Mi), minio/limit-range.yml (namespace minio-minio, default memory limit 2Gi, default request 512Mi memory / 25m CPU), and channel-engine/limit-range.yml (namespace channel-engine, default memory limit 512Mi, default request 256Mi). These three namespaces are exceptions, not the rule; no LimitRange was found for the general population of service or My App namespaces such as valkey-io-valkey or eyevinn-web-runner.

Ephemeral (temporary) storage

Every instance gets a usercontent volume mounted as an emptyDir: {} with no sizeLimit set (osaas-lib-orchestrator/src/instance_service/containers.ts:347-352). Without a sizeLimit, Kubernetes does not cap this volume independently; it is bounded only by the node's available ephemeral storage. No per-tenant ephemeral storage quota was found.

Request size and timeout limits

Two different proxy-body-size values were found, for two different ingress paths. Do not conflate them:

  • The shared per-service auth-gate ingress described above (used by every catalog service instance and every My App, since My App runners are services too) sets nginx.ingress.kubernetes.io/proxy-body-size: '64m' (osaas-lib-orchestrator/src/instance_service/k8s.ts:2045).
  • The platform's own ui application ingress (the web console, app.osaas.io) sets no proxy-body-size override at all, in any environment (infra-osaas/ui/{dev,stage,prod,prod-estx}.yml) — it inherits nginx-ingress's own default of 1m. A separate, unrelated ingress, image-server (namespace gui, host image.svc.{environment}.osaas.io), sets proxy-body-size: "5M" (e.g. infra-osaas/ui/prod-estx.yml:91) — that value belongs to the image-upload backend, not the console or any service instance.

No proxy-read-timeout, proxy-connect-timeout, or proxy-send-timeout override was found in the ingress-nginx-controller ConfigMap (infra-osaas/ingress-nginx.yml:344-395) or on the ingresses inspected for this page. In the absence of an override, the ingress-nginx controller applies its own built-in default, which is 60 seconds for proxy-read-timeout. This has been observed in practice: server actions in osaas-app that poll for longer than roughly 60 seconds are cut off by the ingress before their own timeout logic runs, producing a generic error at the client rather than the application's intended message. If your integration needs long-running synchronous HTTP requests (uploads, long-polling, streaming responses) against a service instance or My App, budget for a 60 second ceiling unless you have separately confirmed a longer timeout is configured for that specific ingress.

Rate limiting and bandwidth quotas

osaas-deploy-manager and osaas-lib-orchestrator both contain a rate-limiting mechanism, but as of the cited commits it is not turned on for any service. The per-service orchestrator deployment optionally reads three environment variables, INGRESS_LIMIT_RPS, INGRESS_LIMIT_BURST_MULTIPLIER, and INGRESS_LIMIT_CONNECTIONS, from optional keys (ingress-limit-rps, ingress-limit-burst-multiplier, ingress-limit-connections) on that service's own {serviceName}-orchestrator ConfigMap (osaas-deploy-manager/src/deploy_service/provider.ts:1780-1809). When set, osaas-lib-orchestrator applies them as nginx.ingress.kubernetes.io/limit-rps, limit-burst-multiplier, and limit-connections annotations on that service's shared ingress (osaas-lib-orchestrator/src/instance_service/k8s.ts:2046-2065).

A repository-wide search of infra-osaas for ingress-limit-rps, ingress-limit-connections, and ingress-limit-burst-multiplier returns zero matches: no service's ConfigMap currently sets these keys. This means the mechanism exists in code but is not configured for any service today, so by default there is no per-instance request-rate or connection limit applied at the ingress. No separate egress bandwidth (data transfer volume) quota mechanism, Cilium/Calico bandwidth annotation, or per-tenant network quota of any kind was found in either infra-osaas or osaas-deploy-manager (searched for limit-rate, limit-connections, limit-req, bandwidth, and the annotation names above, across both repositories).

Summary

Question Answer Source
Is each tenant's service instance in its own Kubernetes namespace? No. One namespace per service, shared across every tenant using that service. osaas-lib-orchestrator/src/instance_service/k8s.ts:1414
Does a Kubernetes NetworkPolicy restrict pod-to-pod traffic between tenants? No. Confirmed both in infra-osaas (no manifest defines one) and live against both production clusters (no cluster-level policy enforces isolation independent of that). Repo-wide grep, zero matches; live cluster verification
Does the auth-gate ingress protect against direct pod-to-pod traffic? No. It only checks requests entering through the external HTTPS ingress. osaas-lib-orchestrator/src/instance_service/k8s.ts:2033-2043
Can a tenant's pod use the Kubernetes API to discover other tenants' resources? No. Confirmed live on both production clusters: the default ServiceAccount tenant pods run under has no permission to list/get pods, namespaces, or secrets outside its own namespace. Live RBAC verification (kubectl auth can-i)
Are CPU/memory limits enforced per tenant container? Not found in the orchestrator code path used by services and My Apps. Three unrelated namespaces have a LimitRange; there is no general one. osaas-lib-orchestrator/src/instance_service/containers.ts, infra-osaas LimitRange search
Is there a request size limit? Yes: 64m on the shared service/My App ingress. The console's own ingress sets no override (nginx-ingress default, 1m). osaas-lib-orchestrator/src/instance_service/k8s.ts:2045, infra-osaas/ui/{env}.yml
Is there a request timeout? No explicit override found; nginx-ingress's default of 60s applies. infra-osaas/ingress-nginx.yml:344-395
Is there a rate limit or bandwidth quota? A per-service opt-in mechanism exists in code but is not configured for any service today. No bandwidth/data-volume quota mechanism was found. osaas-deploy-manager/src/deploy_service/provider.ts:1780-1809, osaas-lib-orchestrator/src/instance_service/k8s.ts:2046-2065

If you have a specific compliance or security question not answered precisely above, ask in the OSC community Slack or open a discussion; this page will be updated as the underlying platform configuration changes.

Clone this wiki locally