Skip to content
Minimal example showing how to enforce HTTP API access control policies using OPA and Envoy on Kubernetes.
Shell Dockerfile
Branch: master
Clone or download
tsandall Update README to use proxy_init:v2
This version does not redirect outbound traffic through
Envoy. For initial access control use cases, outbound traffic doesn't
have to be forced through the proxy and doing so would require more
sophisticated configuration of Envoy.

Signed-off-by: Torin Sandall <torinsandall@gmail.com>
Latest commit 061446f Jul 10, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config Initial commit Jul 8, 2019
policies Initial commit Jul 8, 2019
proxy_init Bypass Envoy for outbound traffic Jul 9, 2019
.gitignore Initial commit Jul 8, 2019
README.md Update README to use proxy_init:v2 Jul 10, 2019

README.md

Minimal OPA-Envoy Sidecar Example for Kubernetes

This repository shows how to run OPA and Envoy as sidecar containers inside an app deployment to enforce HTTP API access control policies. Envoy is statically configured to proxy traffic for the app with the External Authorization filter enabled to use OPA.

Try It Out

These steps have been tested on Kubernetes v1.15.

1. Bootstrap Kubernetes cluster (optional)

Bootstrap a new Kubernetes cluster using kind:

kind create cluster

Set the KUBECONFIG environment variable to point kubectl at the cluster:

export KUBECONFIG=~/.kube/kind-config-kind

2. Create ConfigMaps containing proxy config and app policy

Load an Envoy proxy config into the cluster that passes traffic for the app container and enables External Authorization.

kubectl create configmap proxy-config --from-file config/envoy.yaml

Load an example policy into the cluster that will ALLOW ALL incoming traffic to the app:

kubectl create configmap app-policy --from-file policies/allow_all.rego

In typical deployments the policy would either be built into the OPA container image or it would fetched dynamically via the Bundle API. ConfigMaps are used in this example for test purposes.

3. Create Deployment with sidecar containers

kubectl apply -f example-app-deployment.yaml

example-app-deployment.yaml:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: example-app
  labels:
    app: example-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      initContainers:
        - name: proxy-init
          image: openpolicyagent/proxy_init:v2
          args: ["-p", "8000", "-u", "1111"]
          securityContext:
            capabilities:
              add:
              - NET_ADMIN
            runAsNonRoot: false
            runAsUser: 0
      containers:
        - name: app
          image: python
          args: ["python", "-m", "http.server", "8080"]
        - name: envoy
          image: envoyproxy/envoy:v1.10.0
          securityContext:
            runAsUser: 1111
          volumeMounts:
          - readOnly: true
            mountPath: /config
            name: proxy-config
          args:
          - "envoy"
          - "--config-path"
          - "/config/envoy.yaml"
        - name: opa
          image: openpolicyagent/opa:0.12.1-istio-13
          securityContext:
            runAsUser: 1111
          volumeMounts:
          - readOnly: true
            mountPath: /policies
            name: app-policy
          args:
          - "--plugin-dir=/app"
          - "run"
          - "--server"
          - "--set=plugins.envoy_ext_authz_grpc.addr=:9191"
          - "--set=plugins.envoy_ext_authz_grpc.query=data.system.main"
          - "--ignore=.*"
          - "/policies"
      volumes:
        - name: app-policy
          configMap:
            name: app-policy
        - name: proxy-config
          configMap:
            name: proxy-config

5. Create Service to expose HTTP server

kubectl apply -f example-app-service.yaml

example-app-service.yaml:

kind: Service
apiVersion: v1
metadata:
  name: example-app
spec:
  selector:
    app: example-app
  ports:
  - name: http
    protocol: TCP
    port: 8080
    nodePort: 30808
  type: NodePort

Depending on your Kubernetes cluster you may not want to use a NodePort service. Any kind of service will work for this example.

Get the Service's IP/hostname:

SERVICE_IP=$(kubectl get pod -l app=example-app -o jsonpath='{.items[].status.hostIP}')
echo $SERVICE_IP

6. Test app HTTP endpoint (expect 200)

curl -i $SERVICE_IP:30808

7. Update app-policy ConfigMap

Replace the existing app policy with one that will DENY ALL traffic to the app.

package system

main = false

Delete the app pod to make the update take affect. This will take several seconds while the pod is gracefully terminated.

8. Test app HTTP endpoint (expect 403)

curl -i $SERVICE_IP:30808

How Do I Write Policies?

The input document provided to your policies is defined by the Envoy External Authorization API. For an example of the input document see the open-policy-agent/opa-istio-plugin README.

You can’t perform that action at this time.