Kuzzle cluster mode
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.github
docker-compose
lib
test
.dockerignore
.eslintrc
.gitignore
.travis.yml
CHANGELOG.md
Dockerfile
LICENSE
README.md
dev-npm-install.sh
dev.sh
docker-compose.yml
manifest.json
package-lock.json
package.json
sonar-project.properties

README.md

Build Status codecov.io

Cluster mode plugin

This plugin adds a cluster mode to Kuzzle.

Compatibility

Kuzzle: >=1.4.0

Try it

To run a kuzzle stack, you can use the provided compose file:

docker-compose up --scale kuzzle=3

NB: This compose stack is for tests only and should not be used as-is on production.
Notably, only kuzzle runs in cluster mode, elasticsearch and redis are using one node only.

Run a development stack

The development stack mounts both kuzzle and the cluster sources as docker volumes.

cd <dir>
git clone -b <commit> git@github.com:kuzzleio/kuzzle.git
git clone git@github.com:kuzzleio/kuzzle-plugin-cluster.git

cd kuzzle-plugin-cluster
cp docker-compose/my.env.sample docker-compose/my.env
vim docker-compose/my.env

./dev-npm-install.sh
./dev.sh

You should now have a full Kuzzle clustered stack running 3 Kuzzle front nodes (and 3 servers). Each update on either Kuzzle or the cluster source should automatically restart kuzzle.

Note: on some Linux environments, you may get ENOSPC errors from the filesystem watcher. If so, you need to raise the limits on the number of files that can be watched:

sudo sysctl -w fs.inotify.max_user_watches=524288

nginx vs haproxy

The development stack exposes 2 reverse proxies:

  • nginx on port 7512
  • haproxy on port 7513

haproxy configuration includes some more advanced health checks, which are only partly available in the commercial version of nginx. In counterpart, nginx currently offers a big advantage over haproxy in being able to hot reload its configuration without killing current connections.

In other words, when adding a node to the cluster, haproxy (at least up to current version 1.7) will disconnect all clients, while nginx won't.

Goodies

  • [http://localhost:7575/hastats] (kuzzle/kuzzle) => haproxy stats page
  • [http://localhost:7512/_plugin/cluster/status] => cluster status
  • curl -XPOST http://localhost:7512/_plugin/cluster/reset => resets redis state and force a new sync (blanks cluster state)
  • [http://localhost:7512/cluster_kuzzle_1/] prefixing the url by the container name lets you access it directly

Configuration

Privileged context

This plugin needs privileged context to work. This context is granted by Kuzzle via the global configuration. Add the following to your configuration

plugins: {
    'cluster': {
        privileged: true
    }
}

For more information on how to configure Kuzzle, please refer to the Guide.

Pipe plugin timeouts

This plugin registers some pipe plugins which induce some delay and will exceed default Kuzzle timeouts. Make sure you increase your pipe timeouts accordingly.

  "plugins": {
    "common": {
      "pipeWarnTime": 5000,
      "pipeTimeout": 10000
    }

Redis cluster

Redis cluster comes with some limitations:

  1. Single database only.
  2. Cluster node arrays.

The latter implies the configuration cannot be set via environment variables. To comply with the former, make sure to set only one database index (0).

i.e.:

    "internalCache": {
      "database": 0,
      "nodes": [
        {
          "host": "cluster_redis_1",
          "port": 6379
        },
        {
          "host": "cluster_redis_2",
          "port": 6379
        },
        {
          "host": "cluster_redis_3",
          "port": 6379
        }
      ]
    },
    "memoryStorage": {
      "database": 0,
      "nodes": [
        {
          "host": "cluster_redis_1",
          "port": 6379
        },
        {
          "host": "cluster_redis_2",
          "port": 6379
        },
        {
          "host": "cluster_redis_3",
          "port": 6379
        }
      ]
    }