Skip to content
Scripts to run a dockerized liquid node
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.

Liquid Investigations Node

Scripts and configuration to run a Liquid Node

Build Status


Liquid Node requires a Nomad + Consul + Vault cluster where the software will be deployed. There are a few options:

Whichever option you choose, you will also need to:

  • Increase vm.max_map_count to at least 262144, to make elasticsearch happy - see the docs about elasticsearch in docker for details.
  • Make sure you have Python >= 3.7 installed.

Install a Cluster Manually

Download Consul, Vault and Nomad. For production environments follow their manuals. For development run them all in -dev mode:

./consul agent -dev &
./vault agent -dev &
./nomad agent -dev &

Install the Liquid Cluster

liquidinvestigations/cluster is a self-configuring cluster of Consul + Vault + Nomad. It's optimised for local development, testing, and demo/staging servers.

Use Vagrant

You can run a full Liquid cluster in a local virtual machine using Vagrant. The configuration has been tested with the libvirt driver but should work with the default VirtualBox driver as well.

  1. Install vagrant
  2. cd to the vagrant subfolder, everything Vagrant-related happens in here:
    cd vagrant/
  3. Start the VM:
    vagrant up

You can log into the vm using vagrant ssh. Optionally, install vagrant-env, and set environment variables in an .env local file that will be ignored by git.

Vagrant will forward the following http ports:

  • guest: 80, host: 1380 - public web server
  • guest: 8765, host: 18765 - internal web server
  • guest: 8500, host: 18500 - consul
  • guest: 8200, host: 18200 - vault
  • guest: 4646, host: 14646 - nomad

Vagrant on MacOS

On MacOS the following extra steps need to be taken:

  1. Run the following command in a terminal:
sudo socat -vvv  tcp-listen:80,fork tcp:localhost:1380
  1. Edit /etc/hosts and add the following line:       [liquid_domain] hoover.[liquid_domain]

where [liquid_domain] is the value of liquid.domain from the liquid.ini file.


The Liquid Investigations cluster configuration is read from liquid.ini. Start with the example configuration file:

cp examples/liquid.ini .
vim liquid.ini

We need a way to access Vault. The simplest way is to use:

domain =
debug = true

vault_secrets = /opt/cluster/var/vault-secrets.ini

workers = 1

This assumes the vault-secrets.ini file exists and contains a vault token. It will be generated by the ./ autovault command, or you can create it manually in a different location with the following contents:

root_token = s.Cmro41vNI4wIndgrPqzlqOKY

Then deploy to the cluster:

./liquid deploy

The liquid instance will listen by default on port 80 on the local machine. If you don't have a DNS domain pointing to the macine, you can add entries to /etc/hosts:

Create an initial admin user:

./liquid shell liquid-core ./ createsuperuser


The liquid bundle comes with a versions.ini file with a known set of working versions. You can override them in liquid.ini, see examples/liquid.ini for more information.


Set up the testdata collection. First download the data:

mkdir -p collections
git clone collections/testdata

Next define the collection in liquid.ini:

workers = 1

Then let the deploy command pick up the new collection:

./liquid deploy


Nextcloud runs on the subdomain nextcloud.<liquid_domain>


In order to activate liquid login, follow RocketChatAuthSetup.

Importing collections from docker-setup


The node setup must be clean. For this, either use a new node install (recommended), or remove all collections from liquid.ini and run the following commands:

./liquid purge
./liquid halt
rm -fr ./volumes/hoover


In order to import the collections from docker-setup run the following command:

./liquid importfromdockersetup [path_to_docker_setup] [method]

The method is optional. The default value is link, while the possible values are:

  • link: create links to existing directories in docker-setup
  • copy: copy the data from docker-setup to node
  • move: move the directories from docker-setup to node

When using the copy method, the import will take longer.


Set the debug flag in liquid.ini:

debug = on

Then redeploy (./liquid deploy).

To log into the snoop docker container for testdata:

./liquid shell snoop-testdata-api

To dump the nginx configuration:

nomad alloc fs $(./liquid alloc liquid nginx) nginx/local/core.conf

Working on components

In order to work on Hoover Search, Hoover Snoop, or Liquid Core, first clone the repositories:

cd repos
./ https  # or ./ ssh, based on preference

After that, set this flag in your configuration:

mount_local_repos = true

Running custom jobs

You can deploy your own jobs on the cluster. First, create a nomad job file, you can use one of the existing .nomad files as a starting point. Save it in the local folder, or outside the repository, so that it doesn't interfere with updates. Then add the job to liquid.ini:

template = local/foo.nomad

Afterwards, run ./liquid deploy, which will send your job foo to nomad.

You can’t perform that action at this time.