Kubernetes?

[spacepluk]

Did anybody manage to get this running on Kubernetes?
Thanks

[spacepluk]

This setup is working for me for a very simple deployment:

kind: Service
apiVersion: v1
metadata:
  name: invoiceninja
  labels:
    app: invoiceninja
  annotations:
    external-dns.alpha.kubernetes.io/hostname: yourdomain
spec:
  selector:
    app: invoiceninja
  ports:
    - name: http
      port: 80
      targetPort: http
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: invoiceninja
  labels:
    app: invoiceninja
data:
  nginx.conf: |
    user www-data;

    events {
      worker_connections 768;
    }

    http {
        upstream backend {
            server 127.0.0.1:9000;
        }
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
        gzip on;
        gzip_disable "msie6";

        server {
            listen      80 default;
            server_name _;

            root /var/www/app/public;

            index index.php;

            charset utf-8;

            location / {
                try_files $uri $uri/ /index.php?$query_string;
            }

            location = /favicon.ico { access_log off; log_not_found off; }
            location = /robots.txt  { access_log off; log_not_found off; }

            sendfile off;

            location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass backend;
                fastcgi_index index.php;
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_buffer_size 16k;
                fastcgi_buffers 4 16k;
            }

            location ~ /\.ht {
                deny all;
            }
        }
    }
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: invoiceninja
  labels:
    app: invoiceninja
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: invoiceninja
    spec:
      containers:
        - name: invoiceninja
          image: invoiceninja/invoiceninja
          ports:
            - name: backend
              containerPort: 9000
          env:
            - name: APP_ENV
              value: 'production'
            - name: APP_DEBUG
              value: '0'
            - name: APP_URL
              value: 'yourdomain'
            - name: APP_KEY
              value: 'S0meRandomStringSomeRandomString'
            - name: APP_CIPHER
              value: 'AES-256-CBC'
            - name: TRUSTED_PROXIES
              value: '10.0.0.0/8'
            - name: DB_TYPE
              value: 'mysql'
            - name: DB_USERNAME
              value: 'root'
            - name: DB_PASSWORD
              value: 'ninja'
            - name: DB_HOST
              value: '127.0.0.1'
            - name: DB_DATABASE
              value: 'ninja'
            - name: MAIL_HOST
              value: mailserver
            - name: MAIL_DRIVER
              value: smtp
            - name: MAIL_FROM_NAME
              value: 'InvoiceNinja'
            - name: MAIL_FROM_ADDRESS
              value: 'finance@yourdomain'
          volumeMounts:
            - name: invoiceninja-shared
              mountPath: /var/www/app/public
            - name: invoiceninja-storage
              mountPath: /var/www/app/storage
              subPath: storage
            - name: invoiceninja-storage
              mountPath: /var/www/app/public/logo
              subPath: logo
        - image: mysql:5
          name: mysql
          args: ['--default-authentication-plugin=mysql_native_password']
          env:
          - name: MYSQL_DATABASE
            value: 'ninja'
          - name: MYSQL_ROOT_PASSWORD
            value: 'ninja'
          ports:
          - containerPort: 3306
            name: mysql
          volumeMounts:
            - name: invoiceninja-storage
              mountPath: /var/lib/mysql
              subPath: mysql
        - image: nginx:1
          name: nginx
          ports:
          - name: http
            containerPort: 80
          volumeMounts:
            - name: invoiceninja-shared
              mountPath: /var/www/app/public
            - name: invoiceninja-storage
              mountPath: /var/www/app/storage
              subPath: storage
            - name: invoiceninja-storage
              mountPath: /var/www/app/public/logo
              subPath: logo
            - name: invoiceninja
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
      volumes:
        - name: invoiceninja-shared
          emptyDir: {}
        - name: invoiceninja
          emptyDir: {}
        - name: invoiceninja-storage
          emptyDir: {}
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: invoiceninja
  annotations:
    kubernetes.io/tls-acme: "true"
spec:
  tls:
  - secretName: invoiceninja-tls
    hosts:
    - yourdomain
  rules:
  - host: yourdomain
    http:
      paths:
      - path: /
        backend:
          serviceName: invoiceninja
          servicePort: 80

[lalop]

hello @spacepluk
thank you for sharing this.
Do you know how other project manage to add this in their repository ?

[botzill]

Hi. Thx @spacepluk for sharing this.

How about creating a helm chart with this? It will be super helpful and easy to install and test.

Thx

[spacepluk]

@lalop sorry I didn't see your message. Most projects I've seen just drop the yaml files in a deploy folder.

@botzill that's probably a good idea but I've never used helm myself :)

[botzill]

@spacepluk I'm doing some tests now. Will see how this is going and if all is OK, we can deploy one here: https://github.com/helm/charts. Will be good to have a review from you.

Thx.

[Antipov-Anton]

@botzill What about helm chart for invoiceNinja, you planned to realise? I just research is any chart already available.

[rouja]

Hi,

I just create a helm chart to deploy invoice-ninja on kubernetes. You can find it here. I discovered invoice-ninja few days ago so probably some features are not working. I need feedbacks to improve the chart. So if someone is interested, please feel free to create issues on the repo.

Thx.

[lalop]

Hello, I can't manage the kubernetes part of invoice ninja, but if some of you are okay for maintaining it we may add an official chart to this repos

[spacepluk]

Since this is still the first hit when you google "invoiceninja kubernetes"... I figured I would share my updated config that works with v5 and the alpine images. I hope that helps somebody.

kind: Service
apiVersion: v1
metadata:
  name: invoiceninja
  labels:
    app: invoiceninja
  annotations:
    external-dns.alpha.kubernetes.io/access: public
spec:
  selector:
    app: invoiceninja
  ports:
    - name: http
      port: 80
      targetPort: http
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: invoiceninja-env
  labels:
    app: invoiceninja
data:
  APP_ENV: 'production'
  APP_DEBUG: 'false'
  APP_URL: 'https://invoice.yourdomain.com'
  APP_KEY: 'S0meR4nd0mStr1ngS0meR4nd0mStr1ng'
  APP_CIPHER: 'AES-256-CBC'
  TRUSTED_PROXIES: '*'
  DB_TYPE: 'mysql'
  DB_STRICT: 'false'
  DB_HOST1: '127.0.0.1'
  DB_DATABASE1: 'ninja'
  DB_USERNAME1: 'root'
  DB_PASSWORD1: 'ninja'
  MAIL_FROM_NAME: 'User Name'
  MAIL_FROM_ADDRESS: 'finance@yourdomain.com'
  MAIL_MAILER: 'smtp'
  MAIL_PORT: '587'
  MAIL_ENCRYPTION: 'TLS'
  MAIL_HOST: 'smtp.mailgun.org'
  MAIL_USERNAME: 'username'
  MAIL_PASSWORD: 'password'
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: invoiceninja-cron-config
  labels:
    app: invoiceninja
data:
  cron.sh: |
    #!/bin/sh

    echo "Early Entry"

    cleanup ()
    {
      kill -s SIGTERM $!
      exit 0
    }

    trap cleanup SIGINT SIGTERM

    while :
    do
      sleep 60 ; cd /var/www/app/ && php artisan schedule:run;
    done
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: invoiceninja-nginx-config
  labels:
    app: invoiceninja
data:
  nginx.conf: |
    user www-data;

    events {
      worker_connections 768;
    }

    http {
        upstream backend {
            server 127.0.0.1:9000;
        }
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
        gzip on;
        gzip_disable "msie6";

        server {
            listen      80 default;
            server_name _;

            root /var/www/app/public;

            index index.php;

            charset utf-8;

            location / {
                try_files $uri $uri/ /index.php?$query_string;
            }

            location = /favicon.ico { access_log off; log_not_found off; }
            location = /robots.txt  { access_log off; log_not_found off; }

            sendfile off;

            location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass backend;
                fastcgi_index index.php;
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_param HTTPS 1;
                fastcgi_buffer_size 16k;
                fastcgi_buffers 4 16k;
            }

            location ~ /\.ht {
                deny all;
            }
        }
    }
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: invoiceninja
  labels:
    app: invoiceninja
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: invoiceninja
  template:
    metadata:
      labels:
        app: invoiceninja
    spec:
      initContainers:
      - name: volume-mount-hack
        image: busybox
        command: ["sh", "-c", "chown -R 1500:82 /var/www/app/public /var/www/app/storage"]
        volumeMounts:
        - name: invoiceninja-shared
          mountPath: /var/www/app/public
        - name: invoiceninja-storage
          mountPath: /var/www/app/storage
          subPath: storage
        - name: invoiceninja-storage
          mountPath: /var/www/app/public/storage
          subPath: public-storage
      containers:
        - image: invoiceninja/invoiceninja:5.0.52
          name: invoiceninja
          ports:
            - name: backend
              containerPort: 9000
          envFrom:
          - configMapRef:
              name: invoiceninja-env
          volumeMounts:
            - name: invoiceninja-shared
              mountPath: /var/www/app/public
            - name: invoiceninja-storage
              mountPath: /var/www/app/storage
              subPath: storage
            - name: invoiceninja-storage
              mountPath: /var/www/app/public/storage
              subPath: public-storage
        - image: invoiceninja/invoiceninja:5.0.52
          name: cron
          command: [ 'sh', '/usr/local/bin/cron.sh' ]
          envFrom:
          - configMapRef:
              name: invoiceninja-env
          volumeMounts:
            - name: invoiceninja-cron-config
              mountPath: /usr/local/bin/cron.sh
              subPath: cron.sh
            - name: invoiceninja-shared
              mountPath: /var/www/app/public
            - name: invoiceninja-storage
              mountPath: /var/www/app/storage
              subPath: storage
            - name: invoiceninja-storage
              mountPath: /var/www/app/public/storage
              subPath: public-storage
        - image: mysql:5
          name: mysql
          args: ['--default-authentication-plugin=mysql_native_password']
          env:
          - name: MYSQL_DATABASE
            valueFrom:
              configMapKeyRef:
                name: invoiceninja-env
                key: DB_DATABASE1
          - name: MYSQL_ROOT_PASSWORD
            valueFrom:
              configMapKeyRef:
                name: invoiceninja-env
                key: DB_PASSWORD1
          ports:
          - containerPort: 3306
            name: mysql
          volumeMounts:
            - name: invoiceninja-storage
              mountPath: /var/lib/mysql
              subPath: mysql
            - name: invoiceninja-storage
              mountPath: /data
        - image: nginx:1
          name: nginx
          ports:
          - name: http
            containerPort: 80
          volumeMounts:
            - name: invoiceninja-shared
              mountPath: /var/www/app/public
            - name: invoiceninja-storage
              mountPath: /var/www/app/storage
              subPath: storage
            - name: invoiceninja-storage
              mountPath: /var/www/app/public/storage
              subPath: public-storage
            - name: invoiceninja-nginx-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
      volumes:
        - name: invoiceninja-shared
          emptyDir: {}
        - name: invoiceninja-storage
          persistentVolumeClaim:
            claimName: invoiceninja-data-pvc
        - name: invoiceninja-cron-config
          configMap:
            name: invoiceninja-cron-config
        - name: invoiceninja-nginx-config
          configMap:
            name: invoiceninja-nginx-config
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: invoiceninja-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  #storageClassName: manual
  #volumeName: invoiceninja-data
# ---
# apiVersion: v1
# kind: PersistentVolume
# metadata:
#   name: invoiceninja-data
#   labels:
#     app: invoiceninja
# spec:
#   capacity:
#     storage: 1Gi
#   volumeMode: Filesystem
#   accessModes:
#     - ReadWriteOnce
#   persistentVolumeReclaimPolicy: Retain
#   storageClassName: manual
#   claimRef:
#     name: invoiceninja-data-pvc
#     namespace: invoiceninja # <---------- Make sure this matches your deployment
#   awsElasticBlockStore:
#     volumeID: vol-0123456789abcdef
#     fsType: ext4
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: invoiceninja
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - invoice.yourdomain.com
    secretName: invoiceninja-tls
  rules:
  - host: invoice.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: invoiceninja
            port:
              number: 80

UPDATE: Added the missing cron container (fixes email sending). @turbo124 Thanks for helping!
UPDATE2: Added an example of persistent storage

[Saddamus]

@spacepluk, I have converted your sample to helm template
https://github.com/Saddamus/invoiceninja-helm

[turbo124]

@spacepluk @Saddamus i've referenced this great work in the readme, thankyou!

[justsomebody42]

Hi,
I'd be happy to see an up to date configuration example for Kubernetes 😄
It seems that this example and the referenced helm chart are both not updated anymore?
Is there an "official" page were the most recent version is tracked?

[Saddamus]

Regarding helm chart
I think that the only thing that may need to be updated is imagetag:
tag: "5.0.52", adjusting to latest one from dockerhub. The rest of env should work properly.
If there are some issues, im open to help and update the result on github.

[xeruf]

Seems this is fixed now at https://github.com/invoiceninja/dockerfiles/tree/master/charts/invoiceninja ?
@Saddamus maybe you can link to that from yours as yours still comes up high in search.

[Saddamus]

Seems this is fixed now at https://github.com/invoiceninja/dockerfiles/tree/master/charts/invoiceninja ? @Saddamus maybe you can link to that from yours as yours still comes up high in search.

Done as requested.

[BloodyIron]

I want to add that invoice ninja in docker has been painfully slow for me (v5) for any version higher than v5.2.19. This is with a blank install, as in blank container, blank database, etc. The initial page load for v5.2.19 is quick as one would expect.... v5.3-5.4 default pages are non-usable (http links in all-https page), and v5.5+ (including v5.7.x) take 2+ SECONDS to load the page initially and if anyone refreshes.

So for anyone reading this in the future, use v5.2.19 until the devs figure out the ridiculous performance issues...

[BloodyIron]

Even adding things like "IS_DOCKER" defined as true (env variable) I see no difference... @spacepluk is the latest v5 slow for you too for initial page loads and page refreshes? (f5) and if not... what did you do to speed it up? Again nothing higher than v5.2.19 loads quickly for me no matter what I seem to throw at it so far... v5.2.19 loads very quickly.

[BloodyIron]

btw thanks to @spacepluk and others in this thread, there's almost no kubernetes info for this docker image, the devs don't seem concerned about accurate documentation either, so this information has been utterly crucial to get operational, despite the load issues I still have (keeping me at v5.2.19).

[BloodyIron]

So in the end I have to go with latest since my v4.5.50 will NOT migrate data to v5.2.19 >:| so I still have to deal with the obnoxious initial/refresh page speed... fortunately once it's loaded it's rather responsive.

I would like to add that these were some important details for success for my case:

Notice the change for the group id to 33:
- name: invoiceninja-hermes-volume-mount-hack
image: busybox
command: ["sh", "-c", "chown -R 1500:33 /var/www/app/public /var/www/app/storage"]
volumeMounts:

The underlying permanent storage is using smb.csi.k8s.io the SMB CSI from them, and these the relevant PV mount parameters I use, matching above too:
mountOptions:
- dir_mode=0770
- file_mode=0770
- uid=1500
- gid=33
- vers=3.0

Also note that TRUSTED_PROXIES="*" is a bad practice as it can lead to external websites impersonating to intercept credentials through embedding. Go look up Content Security Policy recommendations to see why this is an insecure configuration. I have it set to my LAN IP range very wide, since this runs in k8s and all IPs are dynamic in there (and I don't manage them at all, by choice).

I had the red triangle until I ran the update secret URL thing, and then refreshed the page

I'm also running mariadb in another deployment, instead of as a sidecar within the pod. That's more my choice really than a requirement for success.

The permission details above corrected the problem of not being able to see company logos. Now I can see them yay! And you will need to have that set correctly BEFORE you migrate any company data form v4! Otherewise you may have to purge company data and try migrating again (TAKE BACKUPS OF COURSE OF THE SOURCE SIDE!)

But yeah, initial page loads are like 4-7 seconds still...