Skip to content

Deploy to a k8s namespace #144

@dboreham

Description

@dboreham

Currently we create "deployment-name-qualified" k8s objects in the default namespace. This allows two or more stacks with the same containers, etc to be deployed into the same cluster without colliding. However it turns out it would be better to use the k8s namespace feature for this. Specifically we had to stop deployment-qualifying "service" object names because that led to their names not being well-known in the cluster. So now we have service names that are not qualified and will collide if the same stack is deployed twice. The solution is to use namespaces, with the namespace having the deployment name and then all the objects we create need to not have qualified names (services have already been changed to this naming scheme).

It's ok to keep the current approach for the "app" name (which includes the deployment id) since that's not exposed anywhere outside the k8s objects.

We will need to arrange to create the namespace, and ensure that all the k8s objects we create go into that namespace. Then we should delete the namespace upon deployment stop, like all the other objects are deleted.

Here's an example of the current naming scheme:

--- PersistentVolumeClaims ---
metadata:
  labels:
    app: stack-45b7fbbede9d821e
    volume-label: stack-45b7fbbede9d821e-db-data
  name: stack-45b7fbbede9d821e-db-data
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2000M

--- Deployments ---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stack-45b7fbbede9d821e-deploy-backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: stack-45b7fbbede9d821e
  template:
    metadata:
      labels:
        app: stack-45b7fbbede9d821e
        service: backend
    spec:
      containers:
      - env:
        - name: DB_HOST
          value: db
        - name: DB_USER
          value: postgres
        - name: DB_PASSWORD
          value: password
        - name: DB_NAME
          value: todoapp
        - name: REACT_APP_API_URL
          value: https://test-todo-1.bpi.servesthe.world/api/todos
        - name: STACK_SVC_BACKEND
          value: backend.default.svc.cluster.local
        - name: STACK_SVC_FRONTEND
          value: frontend.default.svc.cluster.local
        - name: STACK_SVC_DB
          value: db.default.svc.cluster.local
        image: registry.digitalocean.com/stack-testing/todo-backend:deploy-de9d821e
        imagePullPolicy: Always
        name: backend
        ports:
        - containerPort: 5000
        resources:
          limits:
            cpu: '1.0'
            memory: 2000M
          requests:
            cpu: '0.1'
            memory: 200M
        securityContext:
          privileged: false
        volumeMounts: []
      hostAliases:
      - hostnames:
        - backend
        - backend.local
        ip: 127.0.0.1
      imagePullSecrets:
      - name: stack-image-registry
      volumes: []

apiVersion: apps/v1
kind: Deployment
metadata:
  name: stack-45b7fbbede9d821e-deploy-frontend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: stack-45b7fbbede9d821e
  template:
    metadata:
      labels:
        app: stack-45b7fbbede9d821e
        service: frontend
    spec:
      containers:
      - env:
        - name: REACT_APP_API_URL
          value: https://test-todo-1.bpi.servesthe.world/api/todos
        - name: STACK_SVC_BACKEND
          value: backend.default.svc.cluster.local
        - name: STACK_SVC_FRONTEND
          value: frontend.default.svc.cluster.local
        - name: STACK_SVC_DB
          value: db.default.svc.cluster.local
        image: registry.digitalocean.com/stack-testing/todo-frontend:deploy-de9d821e
        imagePullPolicy: Always
        name: frontend
        ports:
        - containerPort: 3000
        resources:
          limits:
            cpu: '1.0'
            memory: 2000M
          requests:
            cpu: '0.1'
            memory: 200M
        securityContext:
          privileged: false
        volumeMounts: []
      hostAliases:
      - hostnames:
        - frontend
        - frontend.local
        ip: 127.0.0.1
      imagePullSecrets:
      - name: stack-image-registry
      volumes: []

apiVersion: apps/v1
kind: Deployment
metadata:
  name: stack-45b7fbbede9d821e-deploy-db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: stack-45b7fbbede9d821e
  template:
    metadata:
      labels:
        app: stack-45b7fbbede9d821e
        service: db
    spec:
      containers:
      - env:
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          value: password
        - name: POSTGRES_DB
          value: todoapp
        - name: REACT_APP_API_URL
          value: https://test-todo-1.bpi.servesthe.world/api/todos
        - name: STACK_SVC_BACKEND
          value: backend.default.svc.cluster.local
        - name: STACK_SVC_FRONTEND
          value: frontend.default.svc.cluster.local
        - name: STACK_SVC_DB
          value: db.default.svc.cluster.local
        image: postgres:14
        imagePullPolicy: Always
        livenessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - !!python/object/new:ruamel.yaml.scalarstring.DoubleQuotedScalarString
              - pg_isready -U postgres
          failureThreshold: 5
          initialDelaySeconds: 0
          periodSeconds: 10
          timeoutSeconds: 5
        name: db
        ports:
        - containerPort: 5432
        resources:
          limits:
            cpu: '1.0'
            memory: 2000M
          requests:
            cpu: '0.1'
            memory: 200M
        securityContext:
          privileged: false
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: db-data
          readOnly: false
      hostAliases:
      - hostnames:
        - db
        - db.local
        ip: 127.0.0.1
      imagePullSecrets:
      - name: stack-image-registry
      volumes:
      - name: db-data
        persistentVolumeClaim:
          claimName: stack-45b7fbbede9d821e-db-data

--- Services ---
metadata:
  labels:
    app: stack-45b7fbbede9d821e
    service: backend
  name: backend
spec:
  ports:
  - name: backend-5000
    port: 5000
    targetPort: 5000
  selector:
    app: stack-45b7fbbede9d821e
    service: backend
  type: ClusterIP

metadata:
  labels:
    app: stack-45b7fbbede9d821e
    service: frontend
  name: frontend
spec:
  ports:
  - name: frontend-3000
    port: 3000
    targetPort: 3000
  selector:
    app: stack-45b7fbbede9d821e
    service: frontend
  type: ClusterIP

metadata:
  labels:
    app: stack-45b7fbbede9d821e
    service: db
  name: db
spec:
  ports:
  - name: db-5432
    port: 5432
    targetPort: 5432
  selector:
    app: stack-45b7fbbede9d821e
    service: db
  type: ClusterIP

--- Ingress ---
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: 'true'
    cert-manager.io/cluster-issuer: letsencrypt-prod
  name: stack-45b7fbbede9d821e-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: test-todo-1.bpi.servesthe.world
    http:
      paths:
      - backend:
          service:
            name: backend
            port:
              number: 5000
        path: /api/todos(/?)(.*)
        pathType: ImplementationSpecific
      - backend:
          service:
            name: frontend
            port:
              number: 3000
        path: /()(.*)
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - test-todo-1.bpi.servesthe.world
    secretName: stack-45b7fbbede9d821e-tls

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions