Skip to content
This repository has been archived by the owner. It is now read-only.
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

scraperwiki / custard

A platform for tools that do stuff with data.

Together with cobalt, custard powers the Sensible Code Company's QuickCode platform.

AGPL Licenced (see LICENCE file).

Initial setup (mongo, nvm)

### On Debian or Ubuntu:

sudo apt-get install -y mongodb

You'll need a bunch of node stuff which we get using nvm:

curl | sh

nvm must be runnable, so add this line to your .profile:

. ~/.nvm/

start a new bash terminal (in order to get nvm which is a shell function)

nvm install 0.10

### On Mac OSX:

Install Node via nvm:

curl | sh
nvm install 0.10

Install Mongo via Homebrew:

brew install mongodb

Then on all platforms:

Clone this repo (inside ~/sw preferable) and from within the custard directory:

mkdir mongo
npm install pow-mongodb-fixtures -g
mongod --dbpath=mongo # might be running already

Alongside custard, you will need to git clone swops-secret

git clone blah blah blah

Automatically getting the correct environment with direnv

direnv can be used to automatically "activate" the environment when you enter the directory. Briefly:

# (install go and set up your GOPATH and PATH sensibly)
go get
go install

# append direnv setup to your bash profile
echo 'eval "$(direnv hook $0)"' >> ${HOME}/.bashrc

# Then cd to the custard directory and enable the .envrc with direnv:
:~$ cd sw/custard/
.envrc is not allowed

:~/sw/custard$ direnv allow
direnv: loading ~/sw/.envrc
connect-assets not found, please run npm install

The first time and then every now and then (npm updates)

# cd into the custard directory
. activate
npm install

Every time you need to develop custard

If you have foreman installed (try gem install foreman) you can go:

foreman start

If you don't have foreman installed you need to do a few things by hand:

# Start a web server (best done in a new window)
. activate # don't need to do this again if you've already done it in this terminal
cake dev

The cake dev above starts custard the web server. It's listening on a local port (usually 3001), so you should be able to visit localhost:3001 in a web browser.


We love them.

First, download Selenium Server and download and unzip Chromedriver, and put them in your custard directory or the next level up:

You need to unzip chromedriver, but not selenium.

If you're wondering about versions, right now we're having luck with:

Then start the Selenium server using cake se:

. activate && cake se

(this is a shortcut for running java -jar selenium-server-standalone-2.*.0.jar

To run the tests:

. activate && cake dev       # runs a local webserver, as above

Or one of these:

mocha test/unit
mocha test/integration

Optional: running Selenium on a Windows machine

On the Windows machine

  • Make sure Java is installed
  • Download the selenium-server jar file (from
  • Download, unpack and add to your path the IE driver (avaiable on the same download page)
  • Run Selenium Server using the command java -jar selenium-server-standalone-2.*.0.jar
  • Run ipconfig in a cmd window to find out the IP address (e.g.

On your local machine

  • export SELENIUM_HOST=
  • export SELENIUM_PORT=4444
  • export BROWSER=iexplorer
  • export CU_TEST_SERVER=<local ip address>
  • run mocha as above and it will use the Windows machine to run the tests

Optional: disabling startup services

You may wish to disable mongodb services from autostarting on boot when not developing custard.

(Tested on Ubuntu 12.04)

Disable mongo service:

echo "manual" | sudo tee /etc/init/mongodb.override

Enable mongo service:

sudo rm /etc/init/mongodb.override


A platform for tools that do stuff with data




No releases published


No packages published