Running Two APISIX Instances in the Same Kubernetes Cluster

[deekshith-hadil]

What I'm Trying to Do

I need HTTP/HTTPS routing and TCP/TLS stream routing in the same Kubernetes cluster. I ended up running two APISIX instances in the same namespace:

1. APISIX-HTTP (L7) — HTTP/HTTPS traffic, managed by the Ingress Controller with CRDs.
2. APISIX-Stream (L4) — TCP/TLS stream traffic, managed directly via the Admin API (no Ingress Controller).

Both share the same etcd but use different key prefixes (/apisix-http and /apisix-stream) so they don't step on each other.

Why Not Just Use CRDs for Stream Routes?

• CRD support for stream routes isn't fully there yet. For example, upstream.timeout (connect, send, read) is silently ignored when applied to stream routes via CRDs — no way to tune TCP timeouts declaratively.
• The Admin API is simpler for dynamic changes — one REST call to update a stream route. No CRD reconciliation loop, no waiting.

Architecture

flowchart LR
    HTTP_Client[HTTP Client] <-->|HTTP/HTTPS| APISIX_HTTP[APISIX-HTTP<br/>L7 Instance]
    APISIX_HTTP <-->|HTTP| HTTP_Server[HTTP Backend]

    TCP_Client[TCP Client] <-->|TCP/TLS| APISIX_STREAM[APISIX-Stream<br/>L4 Instance]
    APISIX_STREAM <-->|TCP/TLS| TCP_Server[TCP Backend]

    IC[Ingress Controller] -->|manages via Admin API| APISIX_HTTP
    Scripts[Automation Scripts] -->|manages via Admin API| APISIX_STREAM

    APISIX_HTTP -->|prefix: /apisix-http| etcd[(Shared etcd)]
    APISIX_STREAM -->|prefix: /apisix-stream| etcd

    style APISIX_HTTP fill:#4da6ff,color:#fff
    style APISIX_STREAM fill:#ff884d,color:#fff
    style etcd fill:#6ec6a0,color:#fff

Loading

Helm Install (Simplified)

APISIX_CHART="apisix/apisix"
APISIX_CHART_VERSION="2.16.0"

# Instance 1: HTTP/HTTPS with Ingress Controller
helm upgrade --install apisix-http "${APISIX_CHART}" \
  --namespace apisix \
  --create-namespace \
  --version "${APISIX_CHART_VERSION}" \
  --set ingress-controller.enabled=true \
  --set ingress-controller.config.kubernetes.ingressClass=apisix-http \
  --set service.stream.enabled=false \
  --set apisix.enableIPv6=false \
  --set apisix.etcd.prefix=/apisix-http

# Instance 2: Stream/TCP with Admin API only
helm upgrade --install apisix-stream "${APISIX_CHART}" \
  --namespace apisix \
  --create-namespace \
  --version "${APISIX_CHART_VERSION}" \
  --set ingress-controller.enabled=false \
  --set service.http.enabled=true \
  --set service.stream.enabled=true \
  --set "service.stream.tcp[0]=5000" \
  --set "service.stream.tcp[1]=5001" \
  --set apisix.enableIPv6=false \
  --set etcd.enabled=false \
  --set "externalEtcd.host[0]=http://apisix-http-etcd:2379" \
  --set externalEtcd.user="" \
  --set apisix.etcd.prefix=/apisix-stream

Question

Are there any issues with running two APISIX instances in a single Kubernetes cluster under one namespace?

Would appreciate any feedback or gotchas. Thanks!

[kayx23]

Running two APISIX releases in the same namespace can work, provided their Kubernetes resources and configuration stores are properly isolated. Whether it is advisable depends on the desired operational and security boundaries; separate namespaces would usually make that isolation clearer.

More importantly, the commands shown do not actually configure separate etcd prefixes. In chart 2.16.0, etcd.prefix is a top-level value, so these should be:

--set etcd.prefix=/apisix-http
--set etcd.prefix=/apisix-stream

The current --set apisix.etcd.prefix=... values are accepted by Helm but are not read by the chart. Rendering 2.16.0 with that flag leaves the generated APISIX configuration on the default prefix: "/apisix" for both releases. Consequently, the two instances can still overwrite configuration under the shared prefix. The chart template reads .Values.etcd.prefix.

Two other points worth considering:

• service.http.enabled=true enables the regular HTTP gateway listener; it is not what enables the Admin API. The Admin API is controlled separately by apisix.admin.enabled, so the HTTP listener can be disabled if the second instance is intended to be stream-only.
• The chart describes its bundled etcd as intended only for development and testing, and recommends connecting to an externally managed etcd installation for production.

After correcting the prefix setting, the shared etcd keyspace should be logically separated, although both instances will still share the availability and operational failure domain of that etcd cluster.