Developer Install

Bo Ferri edited this page Feb 29, 2016 · 25 revisions

These are the installation instructions for developers and will skip some steps described in the [Server-Install](server install) that are not required for development.

Initial Installation


  • currently, we've developed the application only on Ubuntu Linux distributions (empty trusty, 14.04) and OS X. If you experience problems developing on Windows, please contact us.
  • let the $HOME of the less privileged user be '/home/user'
  • all commands boxes start as the less privileged user and in their $HOME.
  • Requiring root is explicitly marked (with su rather than sudo ... ; note that using sudo before each line is not suffienct to yield the same results as with su. Using su may require you to set a password for root if not already done!)

Note: some commands require user input, this is no unattended installation

This step requires root level access

1. install system packages required for building the software

apt-get install --no-install-recommends --yes git-core maven nodejs npm build-essential

These steps require less privileged access

2. create ssh key

ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -N ''

3. add ssh key to to your github user profile


4. clone repositories (not as root!)

lookout for the correct path (/home/user)

cd /home/user

git clone --branch builds/unstable
git clone --branch master
git clone --branch builds/unstable

These steps require root level access

5. install Java for the backend

  • D:SWARM requires Java 8, which is no longer available in the default package sources. Follow these steps
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer oracle-java8-set-default

You can verify your java version with

java -version 2>&1 | grep -q "1.8" && echo "OK, Java 8 is available" || echo "Uh oh, Java 8 is not available"

6. install system packages required for running the software

apt-get install --no-install-recommends --yes mysql-server curl

7. install Neo4j

currently, we rely on Neo4j version 2.3.2

wget -O - | apt-key add -
echo 'deb stable/' > /etc/apt/sources.list.d/neo4j.list
apt-get update
apt-get install --no-install-recommends --yes neo4j=2.3.2

You can open the Neo4j Browser at http://localhost:7474/browser/ to check that the correct version has been installed.

Make sure Neo4j does not get updated when updating packages. You can use apt-pinning to do so. As root, create a file

touch /etc/apt/preferences.d/neo4j.pref

and add the following lines to this file.

Package: neo4j
Pin: version 2.3.2
Pin-Priority: 1000

8. make sure, permissions are correctly

If you didn't change the default places where mysql and neo4j store their data, this step should not be necessary. If you still think you need to set the rights, the default places should be "/var/lib/mysql" and "/var/lib/neo4j/data" )

chown -R mysql:mysql <mysql-data-directory>
chown -R neo4j:adm <neo4j-data-directory>

9. install build environment for frontend

ln -s /usr/bin/nodejs /usr/bin/node
npm install -g grunt-cli karma bower

10. setup MySQL

Create a database and a user for d:swarm. To customize the settings, edit dswarm/persistence/src/main/resources/create_database.sql. Do not check in this file in case you modify it. Hint: remember settings for step 12 (configure d:swarm).

mysql -uroot -p < persistence/src/main/resources/create_database.sql

Then, open /etc/mysql/my.cnf and add the following line to the section [mysqld] (around line 45)

wait_timeout = 1209600

11. setup Neo4j

increase file handlers at /etc/security/limits.conf

root   soft    nofile  40000
root   hard    nofile  40000

plus add ulimit -n 40000 into your neo4j service script (under /etc/init.d', e.g., /etc/init.d/neo4j-service`) before starting the daemon

edit /etc/neo4j/ and:

  • insert some storage tweaks

edit /etc/neo4j/ and

  • at line 24, disable authentication
  • at line 75 and add our graph extension
  • (optional) specify IP address

edit /etc/neo4j/neo4j-wrapper.conf and:

  • insert an additional parameter (if your server is x64)
  • tweak the java heap space size to an appropriate value according to your server ram memory, e.g.,

By default, the Neo4j Server is bundled with a Web server that binds to host localhost on port 7474, answering only requests from the local machine.

These steps require less privileged access

12. configure d:swarm

Place a file named dswarm.conf somewhere in your filesystem and put in the following content

cat <<EOF>>dswarm.conf

Username and password are used to access the MySQL database (hint: use same as in step 9 Setup MySQL) You should not put this file under version control. Name it dswarm.conf and place it in the root directory of the project dswarm. This file is ignored by git.

More information about configuration you get here: d:swarm Configuration.

13. build neo4j extension

pushd dswarm-graph-neo4j
mvn -U -PRELEASE -DskipTests clean package
mv dswarm-graph-neo4j/target/graph-1.2-jar-with-dependencies.jar dswarm-graph-neo4j.jar

14. build backend

pushd dswarm
mvn -U -DskipTests clean install -Dconfig.file=/path/to/dswarm.conf

note: Please specify the path to your custom d:swarm config, if it is not located at the root directory of the d:swarm backend repository. Otherwise, you can run the maven task with argument -Pdswarm-conf (which looks at the root directory of the d:swarm backend repository for a d:swarm config named dswarm.conf)

15. build frontend

pushd dswarm-backoffice-web; pushd yo
npm install
bower install
STAGE=unstable DMP_HOME=../../dswarm grunt build
rsync --delete --verbose --recursive yo/dist/ yo/publish

note: npm install may needs to be executed as root

These steps require root level access

16. wire everything together

lookout for the correct path (/home/user)

cp /home/user/dswarm-graph-neo4j.jar /usr/share/neo4j/plugins/

17. restart everything, if needed

/etc/init.d/mysql restart
/etc/init.d/neo4j-service restart

18. initialize/reset database

This step requires less privileged access

When running the backend the first time, the MySQL database needs to be initialized. When updated, a reset is required in case the schema or initial data has changed. lookout for the correct path (/home/user)

pushd dswarm/dev-tools
python \
  --persistence-module=../persistence \
  --user=dmp \
  --password=dmp \
  --db=dmp \

Or provide the credentials and values you configured. Check python --help for additional information.

Update the System

For now , please refer to the Server-Install for updating the system.

Checklist on Errors

For now , please use the checklist from the Server-Install. Please keep in mind that the links used there will take you to the server install not to the developer version!

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.