Full install

docker-compose/aio/.env

@@ -0,0 +1,2 @@
+NC_INSTALL_ROOT=./
+NO_COLOR=NEST_JS_LOG_MESSAGE_NO_COLOR_SET_NON_NULL_VALUE

docker-compose/aio/.gitignore

@@ -0,0 +1,3 @@
+conf/
+data/
+logs/

docker-compose/aio/README.md

@@ -0,0 +1,71 @@
+# Install all-in-one nocodb with Docker (compose)
+
+This page provides instructions to install nocodb all-in-one (aio) using Docker-Compse. The installation will run multiple contianers in single node which includes
+- nocodb
+- postgres
+- nginx
+- redis
+
+## Prerequisites
+Before you begin, ensure you have the following prerequisites:
+
+- Docker (version 20.10.7 or later)
+- Docker-Compose (version 2.17.3 or later)
+- Ports 80 and 443 are available 
+
+TIP: you could simply run [./pre-req-check.sh](./pre-req-check.sh) which performs pre-requisite check.
+
+## Install 
+Run [install.sh](./install.sh), This script performs pre-requisite check, prompts you through required application properties and finally performs `docker-compose up -d`. 
+For most cases where no external integration required. The defaults properties are just fine. 
+```
+sudo ./install.sh 
+```
+Note: sudo is required for docker to run unless you have configured docker user to be part of sudoers. If sudo is not used then you will get error `('Connection aborted.', PermissionError(13, 'Permission denied'))`
+
+* At this point, your installation is completed and you should be able to access your nocodb instance *
+
+### An example output will be like below. 
+```
+./install.sh 
+** Performing nocodb system check and setup. This step may require sudo permissions
+ | Checking if required tools (docker, docker-compose, jq, lsof) are installed...
+ | Checking port accessibility...
+ | Port 80 is free.
+ | WARNING: Port 443 is in use. Please make sure it is free.
+** System check completed successfully. **
+** Few pre-requisites are failing. Recommend to resolve and proceed. However you could still proceed to install **
+ | Press Y to continue or N to skip (Y/N)? 
+Preparing environment file before install..
+ | Press Y to continue with defaults or N to customise app properties (Y/N)
+Backing up previous docker-compose/aio/conf/nc_properties.env file to nocodb/docker-compose/aio/conf/nc_properties.env-1707455571.bak
+Environment variables written to docker-compose/aio/conf/nc_properties.env file.
+Installing docker containers
+```
+
+## Data and Conf directories
+This directory acts as the NC_INSTALL_ROOT by default and it will have data, conf directories which are `.gitingore` to avoid accidentlly exposing to git repository. 
+During installation the default properties are configured at [nc_properties.env](./conf/nc_properties.env) which can be updated if required and restarted 
+
+```
+.
+├── conf
+│   └── nc_properties.env
+├── data
+│   ├── nginx
+│   ├── nocodb
+│   ├── postgres
+│   └── redis
+├── docker
+│   └── docker-compose.yml
+```
+
+
+## what does install.sh do
+[Install script](./install.sh) performs the following steps
+1. pre-req-check.sh and warns if there is anything missing which could potentially cause issues at later stage. However it will let you proceed if you wish to.
+2. create application properties file under conf dir which will then be used for future upgrades etc.
+3. runs docker-compose up -d
+
+## Advanced Operations
+Refer [advanced section](./advanced.md) for advanced operations like setting up ssl, updating configurations, restarts etc

docker-compose/aio/advanced.md

@@ -0,0 +1,47 @@
+# Advanced operations 
+
+## Restarting containers 
+There are atleast 4 main containers which are running as part of this installation through same docker-compose. The same docker-compose can be leveraged to restart any or all of these containers. 
+
+Use [restart.sh](./bin/restart.sh) or Use below command to restart all containers
+``` docker compose restart ```
+
+To restart individual containers with name ( names:  nocodb, nginx, postgres, redis)\
+ex: to restart nginx\
+``` docker compose restart nginx ```
+
+## Reload nginx 
+use utility script at [./bin/nginx_reload.sh](./bin/nginx_reload.sh)
+
+## [TBD] Upgrade nocodb instance 
+
+## Enable SSL
+To enable SSL for incoming https requests, nginx should be configured with combination of a public certificate and a private key. The SSL private key is kept secret on the server. It will be used to encrypt content sent to clients. 
+Below are different approaches to get and configure certificates. Make your choice
+### letsencrypt for generating certificates 
+Certificates/key can be obtained by trusted CA (Certificate Authorities), there are many paid vendors found online or you can also use [letsencrypt](https://letsencrypt.org/) a non profit certificate provider for free however we recommend [donating](https://www.abetterinternet.org/donate/) for their work.
+
+Run the script to create certificate using the script 
+```
+./bin/gen_letsencrypt_cert.sh 
+```
+
+### [TBD] Bring your own certificates 
+If you already have the certificates, either self signed or generated by any other means, you will need to configure them with nginx. Below are the steps
+
+### [TBD] Self signed certificates 
+One of the pre-requisite is that your server should be associated with the domain name. In the absence of that you could use self signed certificates which does ecrypt but browsers show warning.
+
+## Database password rotation 
+As a security measure, It is best practice to rotate the database credentials periodically. Assuming you would have created new credentials in postgres database. The db credentials are persisted on filesystem as part of initial install and it will be available at 
+[./conf/nc_properties.env](./conf/nc_properties.env)\
+update properties POSTGRES_USER, POSTGRES_PASSWORD with new credentials and [restarting nocodb](#restarting-containers) with\
+```docker compose restart nocodb```
+
+## nginx configurations 
+There are two main directories where nginx configurations are maintained 
+- nocodb team managed configurations at [nginx/conf.d](./conf/nginx/conf.d).
+- self managed (you) [conf/nginx/conf.d](./conf/nginx/conf.d) 
+
+## postgres configurations 
+[postgres.conf](./data/postgres/postgresql.conf) and [pg_hba.conf](./data/postgres/pg_hba.conf) are created under ./data/postgres directory upon first postgres container creation. The configurations can be updated and restarted continer to take affect.

docker-compose/aio/bin/gen_letsencrypt_cert.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# expects nginx to be up and running with conf.d/certbot.conf
+# dns to be mapped to the machine where cert is generated
+#
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+SERVER_NAME=${1}
+if [[ -z "$SERVER_NAME" ]]
+then
+    echo "required argument servername"
+    echo "usage ex: ./gen_certs my.nocodb.com"
+    exit 1
+fi
+
+echo "Creating configs for SERVER_NAME: ${SERVER_NAME}"
+cd ${SCRIPT_DIR}/../conf/nginx/conf.d
+sed "s,<SERVER_NAME>,${SERVER_NAME},g" ${SCRIPT_DIR}/../nginx/conf-templates/certbot_conf.template > certbot.conf
+
+cd ${SCRIPT_DIR}/../bin
+./nginx_start.sh 
+./nginx_reload.sh 
+
+echo "Triggering certbot to create ssl configs: ${SERVER_NAME}"
+cd ${SCRIPT_DIR}/..
+docker-compose run --rm  certbot certonly --webroot --webroot-path /var/www/certbot/  -d ${SERVER_NAME} 
+result=$?
+
+if [[ $result == 1 ]]; then
+    echo "cert generation failed"
+    echo "rolling back the certs and reloading nginx"    
+else
+    echo "Now reload nginx with new ssl configs for your site : ${SERVER_NAME}"
+    cd ${SCRIPT_DIR}/../conf/nginx/conf.d
+    rm -f certbot.conf 
+    sed "s,<SERVER_NAME>,${SERVER_NAME},g" ${SCRIPT_DIR}/../nginx/conf-templates/ssl_server_name_conf.template > ${SERVER_NAME}.conf
+fi    
+rm -rf ${SCRIPT_DIR}/../conf/nginx/conf.d/certbot.conf
+${SCRIPT_DIR}/../bin/nginx_reload.sh

docker-compose/aio/bin/nginx_reload.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+docker exec -it nginx /etc/init.d/nginx reload

docker-compose/aio/bin/nginx_start.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+# starts the docker containers configured in this components 
+# docker compose dir
+#
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+COMPONENT_DIR=${SCRIPT_DIR}/../
+cd ${COMPONENT_DIR}
+mkdir -p ${COMPONENT_DIR}/data
+chmod -R 777 ${COMPONENT_DIR}/data
+docker-compose restart nginx
+

docker-compose/aio/bin/renew_certs.sh

@@ -0,0 +1,3 @@
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+cd ${SCRIPT_DIR}/../
+sudo docker-compose run --rm certbot renew -q

docker-compose/aio/bin/restart.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+# docker-compose restart all containers utilty script
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+COMPONENT_DIR=${SCRIPT_DIR}/../
+cd ${COMPONENT_DIR}
+docker-compose restart 

docker-compose/aio/bin/start.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+# docker-compose start 
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+COMPONENT_DIR=${SCRIPT_DIR}/../
+cd ${COMPONENT_DIR}
+docker-compose up -d 

docker-compose/aio/conf/nc_properties.env

@@ -0,0 +1,13 @@
+# Environment Variables
+POSTGRES_USER=postgres
+POSTGRES_PASSWORD=test123
+POSTGRES_DB=nocodb
+NC_REDIS_URL=redis://redis:6379/4
+NC_DB=pg://postgres:5432?u=postgres&password=${POSTGRES_PASSWORD:-nocodb}&d=postgres
+NC_PUBLIC_URL=http://rajanishs-MacBook-Pro.local
+NC_CONNECT_TO_EXTERNAL_DB_DISABLED=false
+NC_INVITE_ONLY_SIGNUP=false
+NC_ADMIN_EMAIL=admin@nocodb.com
+NC_ADMIN_PASSWORD=nocodb123
+NC_S3_BUCKET_NAME=nocodb-attachements
+

docker-compose/aio/docker-compose.yml

@@ -0,0 +1,105 @@
+version: '3.8'
+
+networks:
+  nocodb-001:
+    # external: true
+
+services:
+  redis: 
+    image: redis:latest
+    container_name: redis
+    restart: unless-stopped
+    env_file:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nc_properties.env
+    expose:
+      - "6379"    
+    volumes:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/data/redis:/data
+    networks:
+      - nocodb-001
+    deploy:
+      resources:
+        limits:
+          cpus: '0.5'
+          memory: 1000M      
+
+  postgres:
+    image: postgres:14.7
+    container_name: postgres
+    restart: unless-stopped
+    env_file:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nc_properties.env
+    expose:
+      - "5432"              
+    volumes:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/data/postgres:/var/lib/postgresql/data    
+    networks:
+      - nocodb-001
+    healthcheck: 
+      interval: 10s
+      retries: 10
+      test: "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"
+      timeout: 2s      
+    deploy:
+      resources:
+        limits:
+          cpus: '1'
+          memory: 1000M      
+
+  nocodb:
+    depends_on:
+      - postgres
+      - redis
+    image: nocodb/nocodb:latest
+    container_name: nocodb
+    restart: unless-stopped
+    env_file:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nc_properties.env
+    expose:
+      - "8080"                 
+    volumes:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/data/nocodb:/usr/app/data/      
+    networks:
+      - nocodb-001
+    deploy:
+      resources:
+        limits:
+          cpus: '1'
+          memory: 1000M
+
+  nginx:
+    container_name: nginx
+    depends_on:
+      - nocodb
+    image: nginx
+    restart: unless-stopped
+    env_file:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nc_properties.env    
+    volumes:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/nginx/conf.d:/etc/nginx/conf.d:ro
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nginx/conf.d:/etc/nginx/custom-conf.d:ro
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/nginx/conf:/opt/nocohub/nginx/conf
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/nginx/conf/nginx.conf:/etc/nginx/nginx.conf:ro
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/data/nginx:/opt/nocohub/nginx/data
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nginx/certbot/www:/var/www/certbot/:ro
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nginx/certbot/conf/:/etc/nginx/ssl/:ro      
+      # -  ../nginx/conf/ssl:/etc/nginx/ssl/:ro   
+    expose:
+      - "80"                 
+      - "443"                 
+    ports:
+      - "80:80"
+      - "443:443"
+    networks:
+      - nocodb-001
+    deploy:
+      resources:
+        limits:
+          cpus: '1'
+          memory: 1000M    
+  certbot:
+    container_name: nocodb_certbot
+    image: certbot/certbot:latest
+    volumes:
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nginx/certbot/www:/var/www/certbot/:rw
+      - ${NC_INSTALL_ROOT:-/opt/nocodb}/conf/nginx/certbot/conf/:/etc/letsencrypt/:rw           

docker-compose/aio/install.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+# Performs Initial setup and System Requirements Check 
+
+## 1. validate system requirements
+# a. docker, docker-compose, jq installed 
+# b. port mapping check 
+#   - port 80,443 are free or being used by nginx container
+#   - port 8080 is open if used for multi-instance setup
+#   - port 6379 for redis access
+#   - port 9001 for minio access 
+# c. docker repo accessiblity quay.io/minio/minio:RELEASE.2023-12-09T18-17-51Z, redis:latest, postgres:14.7, nocodb/nocodb:latest, nginx
+# d. licence check (tbd)
+
+# -- main line code starts here
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+## utility functions 
+source ${SCRIPT_DIR}/sbin/util.sh
+
+${SCRIPT_DIR}/pre-req-check.sh
+PRE_REQ_SUCCESS=$?
+if [[ ${PRE_REQ_SUCCESS} != 0 ]]
+then
+  echo "** Few pre-requisites are failing. Recommend to resolve and proceed. However you could still proceed to install **" >&2
+else
+  echo "** All pre-requistites are taken care of. Proceeding to install.. **" 
+fi  
+
+# ask do you want to proceed with all defaults,
+# if yes, then no prompts 
+if asksure; then
+    echo "Preparing environment file before install.."
+    promptUser=true
+    if asksure " | Press Y to continue with defaults or N to customise app properties (Y/N)"; then
+      promptUser=false      
+    fi
+    ${SCRIPT_DIR}/prepare_env.sh ${promptUser}
+    echo "Installing docker containers"
+    docker-compose -f ${SCRIPT_DIR}/docker-compose.yml up -d
+  else
+    echo "Exiting without install. You can install using docker-compose -f ${SCRIPT_DIR}/docker-compose.yml up -d "
+fi

docker-compose/aio/nginx/conf-templates/certbot_conf.template

@@ -0,0 +1,15 @@
+server {
+    listen 80;
+    listen [::]:80;
+
+    # chantge server_name while generating cert
+    server_name <SERVER_NAME>;
+
+    #access_log  /var/log/nginx/host.access.log  main;
+
+    # this is required for cert generation. 
+    # change server_name as well with cname of required cert
+    location /.well-known/acme-challenge/ {
+        root /var/www/certbot;
+    }                     
+}