New version of the ibex farm written for node using a postgres db instead of the fs
JavaScript CSS Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
deploy
docs
exptemplates
ibexincludes
lib
logo
prepublic
public
scripts
skeleton
test
testcert
views
.gitignore
LICENSE
Makefile
Readme.markdown
TODO
package.json
setenv.sh
webpack.common.js
webpack.config.js
webpack.dll.js

Readme.markdown

Notes on code and architecture

This is a nodejs app written in Javascript. The vast majority of the code is in ES5, since the project started before ES6 support was available, but newer client-side and server-side code is written in ES6.

Permanent application data is stored in a postgres database. Some transient data (principally session info) is stored in a redis database. Nothing is stored directly in memory or on the file system.

The user interface is written using React (http://facebook.github.io/react/). This library makes use of .jsx files, which allow HTML to be interspersed with Javascript code. The site has a single-page design and uses Davis (https://github.com/olivernn/davis.js/) as a layer on top of the HTML5 history API. Links which are not handled by Davis have the data-davis attribute set to no. Client-side Javascript code is loaded on an as-needed basis using RequireJS (http://requirejs.org/). It would be a good idea to move over to a more sophisticated module/package manager for client-side Javascript at some point, since the current setup gives rise to lots of separate requests for individual JS files. However, some of the more popular options assume that you want a single JS blob containing your entire app, and don't cater well for loading code on demand. It might also be an idea to switch from Davis to react-router at some point. Davis seems to do everything needed, but does require a couple of ugly hacks to handle navigation through pages with headers/footers.

The development environment is set up using Docker, as detailed in the next section. The idea is that the app will also be deployed using Docker, but no serious work on deployment has been done yet. In the dev environment, everything runs in a single container. This would probably fine for a small production deployment, but ideally the postgres database, redis database and web server components should run in their own containers. This being said, a single-container deployment would be useful for people who want to run their own instance of Ibex for a single project or lab.

Setting up a test server using docker

Creating the docker containers

The present git repository should be cloned into $DIR/nodeibexfarm on your development machine (your choice of $DIR). $DIR should be otherwise empty.

We begin by creating a data only container for the postgres database. (There is no need to persist the redis database.)

docker create -v /postgresdata --name postgresdata phusion/baseimage /bin/true

The main application container can now be built using the Dockerfile in deploy/:

cd deploy
docker build -t nodeibexfarm-devel .

The container is then started as follows:

docker run --rm -i -p 80:5000 -p 443:6002 -v $DIR:/home/docker/data --volumes-from postgresdata -t nodeibexfarm-devel /sbin/my_init -- /bin/bash -c 'su docker'

The --rm flag deletes the container on exit, which prevents multiple container instances hanging around and wasting disk space. It is good practice to use this flag, since you should not be storing any information within the development container.

For convenience, the preceding command can be executed using the runcontainer.sh script in the deploy dir of this repo:

cd deploy
DATADIR=$DIR sh runcontainer.sh

$DATADIR is mapped to /home/docker/data. This way, you can edit the source code as normal outside of the docker container and the changes will be visible within the docker container.

Most of the time, you can do stuff as the 'docker' user within the container and use sudo to execute commands that require more privileges. The runcontainer.sh script starts you out as the 'docker' user.

Initializing the container and managing services

There are a number of scripts in /home/docker/src/nodeibexfarm/deploy/scripts. Within the container, these can be run using the ibexrun utility in /usr/local/bin. You should begin by running:

ibexrun initenv development

This script:

  • Initializes the postgres DB.

  • Installs required node modules using npm -d.

You only need to run this script once. You can change 'development' to 'production' for a production server (but things should not in any way be considered production ready at this point).

The development environment uses runit to manage running processes. You can start and stop the necessary services as follows. They will start automatically after running ibexrun initdevenv.

sudo sv up redis
sudo sv up postgres
sudo sv up ibexweb
sudo sv up hoover

sudo sv down redis
sudo sv down postgres
sudo sv down ibexweb
sudo sv down hoover

When these services are up, you can connect to the container on port 80 (HTTP, redirects to HTTPS) or on port 443 (HTTPS).

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT

Some notes on server deployment

On CentOS 7, you can redirect ports 80 and 443 to ports 5000 and 6002 as follows:

service firewalld start
firewall-cmd --zone=external --add-forward-port=port=443:proto=tcp:toport=6002
firewall-cmd --zone=external --add-forward-port=port=80:proto=tcp:toport=5000

Getting the IP address to connect to in your browser

Docs assume that you have docker set up so that http://localhost is forwarded to the docker VM.

Logging

Logs are available in the following locations:

  • /home/docker/redis.log
  • /home/docker/postgres.logdir/current
  • /home/docker/ibexweb.logdir/current

Connecting to postgres

Use the follow command to connect to the postgres server:

psql -h localhost -U ibexfarm -d ibexfarm

(For some operations, you may need to connect with -U postgres.)

Wiping the postgres database

Sometimes during development it's useful to wipe the db clean. To do this:

sudo sv stop ibexweb
sudo sv stop hoover
psql -h localhost -U postgres -d postgres
    > DROP DATABASE ibexfarm;
    > CREATE DATABASE ibexfarm;
    > Ctrl+D
psql -h localhost -U ibexfarm -d ibexfarm
    > \i ~/data/nodeibexfarm/scripts/makedb.sql
    > Ctrl+D
cd ~/data/nodeibexfarm
bash mysetenv.sh node scripts/loadexptemplates.js
sudo sv start ibexweb && sudo sv start hoover

Config vars

There is an example configuration file in setenv.sh in the main nodeibexfarm dir. The actual values for the config variables are set in mysetenv.sh. You can edit mysetenv.sh and then restart the ibexweb service to see the new values take effect.

Generating Javascript and CSS files

Javascript and CSS files are built using the Makefile in the main project directory:

make clean
make all

By default, everything is minified, uglified etc. for production. To turn all of this off, set the IBEX_MODE environment variable to "dev":

IBEX_MODE=dev make all

Note that when changing from one value of IBEX_MODE to another, you will need to do a make clean first to ensure that everything is rebuilt in (un)minified form.