Digital Repository of Ireland's Samvera application
Ruby XSLT HTML CSS JavaScript Gherkin Shell
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
config
db
doc
features
hacking
lib
log
public
script
smoothness
solr/config
spec
tasks
test
vendor
.fcrepo_wrapper
.gitignore
.gitmodules
.rspec
.rubocop.yml
.ruby-version
.solr_wrapper
FETCH_HEAD
Gemfile
Gemfile.lock
Guardfile
LICENSE.rdoc
README.rdoc
Rakefile
ansible.cfg
buildshim
config.ru
custom_plan.rb
zeus.json

README.rdoc

Welcome to the DRI Hydra Head

This repository contains the Digital Repository of Ireland's Hydra application.

There are a number of other depedencies (see below) but at this point you will need to have a functional Rails stack and at least Java version 7. You can check your Java version by running 'java -version' anything below 1.7x in the output and you should update from java.com/en/download/

For testing and development you will need to bootstrap the project

$ git clone git@tracker.dri.ie:drirepo/dri-app.git
$ cd dri-app
$ bundle
$ cp config/database.yml.sample config/database.yml (and edit as appropriate)
$ cp config/fedora.yml.sample config/fedora.yml (and edit as appropriate)
$ cp config/jetty.yml.sample config/jetty.yml (and edit as appropriate)
$ cp config/redis.yml.sample config/redis.yml (and edit as appropriate)
$ cp config/solr.yml.sample config/solr.yml (and edit as appropriate)
$ bundle exec rake db:migrate
$ bundle exec rake db:seed

Dont forget to prepare the test database with

$ bundle exec rake db:test:prepare

You may also need to edit config/settings.yml and/or create config/settings.local.yml

To start the application for demo purposes

Note: Ensure that a mysql (or mysql replacement) is running. (E.G. MariaDB start alias: alias mariadbstart='cd “/opt/local” ;sudo /opt/local/lib/mariadb/bin/mysqld_safe –datadir=“/opt/local/var/db/mariadb”')

$ cd dri-app
$ rake jetty:start
$ rake jetty:config
$ bundle exec rails server

The db:seed step above will create 4 initial accounts:

  • admin@dri.ie (admin user)

  • orgmanager@dri.ie (organisation manager)

  • manager@dri.ie (collection manager)

  • user@dri.ie (public user)

The default password for all 3 accounts is set at install time from the password setting in the settings.yml file. Typically this will be set to “CHANGEME” and it should be updated as soon as possible after installation. To change the password log into the application as each of the default users, navigate to their profile page by clicking on the username at the top of the screen, and clicking on the 'Edit' button.

To run the unit tests

$ cd dri-app
$ rake jetty:start
$ rake jetty:config
$ rake rspec

To run the functional tests

$ cd dri-app
$ rake jetty:start
$ rake jetty:config
$ rake cucumber or bundle exec cucumber

Dependencies

If you are using homebrew on OSX

$ brew install ansible coreutils libtool libksba libxml2 libxslt automake autoconf openssl curl curl-ca-bundle readline lzlib gettext pkg-config libyaml mariadb redis clamav sqlite ffmpeg gource phantomjs fits glib imagemagick ghostscript node

In order to properly ingest videos, please reinstall ffmpeg with all the the supported codecs and encoders available:

$ brew reinstall ffmpeg --with-fdk-aac --with-ffplay --with-freetype --with-frei0r --with-libass --with-libvo-aacenc --with-libvorbis --with-libvpx --with-opencore-amr --with-openjpeg --with-opus --with-rtmpdump --with-schroedinger --with-speex --with-theora --with-tools

Testing

The unit tests are written using rspec and the functional tests are written with cucumber.

This project is set up to use guard to automatically run tests as they are changed.

$ cd dri-app
$ bundle exec guard

The poltergeist gem is required for testing, as a result phantomjs binary to run poltergeist.

Translations

When translations are added to the english locale files in config/locales you should run the rake task localefiles with the parameter file which should be set to the path to the ENGLISH yml file. For example:

$ rake locales:mergefile file=config/locales/dri/en.yml

This will merge any new keys from the English file into any other locale files (with the same naming convention) in that directory. The default value for each key will be the English text.

To merge all files use

$ rake locales:mergeall

Settings.yml

This is where the application specific settings are located

config/settings.yml

Please view and edit this file as needed before deploying the application.

To override settings.yml with local settings please use settings.local.yml.

Configuation of database.yml, fedora.yml and solr.yml

The configuration files for the application

  • config/database.yml

  • config/solr.yml

  • config/fedora.yml

are not configured initially. There are some sample configurations in the config directory. These configuration files must be created before development, testing, deployment. The default sample files can be copied into the correct locations for testing and development. It is suggested that these files are not commited to the repository.

Background jobs

The application includes a queueing system based on Resque and Redis to allow asynchronous processing of background jobs. This introduces the following dependencies:

* Redis (server for the queueing system)
* fits (for extracting metadata from a file)
* ffmpeg (video file processing tool)
* imagemagick (image processing)
* Clamav (malware scanning)

For testing, the file processing tools should be installed locally with the hydra-head. See the Dependencies section of this document for more information on installing these and other dependencies.

By default fits should be installed in the directory /opt/local/fits/ (this can be changed by modifying the config.fits_path setting in the file config/initializers/sufia.rb).

By default ffmpeg should be installed in the directory /opt/local/bin (this can be changed by modifying the ffmpeg_path setting in the file config/settings.yml).

For testing purposes Redis can be installed locally and started with the command

$ sudo redis-server /opt/local/etc/redis.conf

The worker queues can be started with the following rake task

$ rake environment resque:work RAILS_ENV=development QUEUE="*" COUNT="2" VERBOSE=1

To view your queues and workers you can visit the path /resque on the web application where resque-web is mounted.

Note that the queuing code is stubbed for the Cucumber tests so you do not need a redis host running in order to run the tests.

ClamAV

To enable virus scanning ClamAV needs to be installed.

$ sudo port install clamav

You will need to create the configuration file freshclam.conf in /opt/local/etc. There is an example file in /opt/local/etc/example-freshclam.conf. Set the directory for the database files and the log

DatabaseDirectory /opt/local/share/clamav
UpdateLogFile /opt/local/var/log/clamav/freshclam.log

Update the virus database

$ sudo freshclam

Install the clamav Ruby gem

$ gem install clamav -- --with-cflags=`clamav-config --cflags` --with-ldflags=`clamav-config --libs`

Add the gem to the GemFile

gem 'clamav'

Run bundler. Virus scanning should now take place when an asset is uploaded.

Storage

Surrogate files are stored using the S3 interface. To create surrogates you will need access to an S3 server. A Ceph VM with S3 Rados Gateway is available in the drirepo repository. To start up Ceph run

$ vagrant up ceph

When the VM is running, log in with

$ vagrant ssh ceph

Then create a user with no max bucket limit as follows:

$ sudo radosgw-admin user create --uid=johndoe --display-name="John Doe" --email=john@example.com --max-buckets=0

This will return an access_key and secret_key which must be added to the hydra head's config/settings.yml or config/settings.local.yml file as follows:

S3:
  server: 10.0.1.201
  access_key_id: 4WP751XIFXBH9G97DD91
  secret_access_key: v6tX8bZSeN9IRJAcelHLnQFrsYV8ojxgWa7rn0uz

Note that the keys should be pasted in as they appear in the output of the radosgw-admin command, but all '' characters should be stripped.

An alternative, if the Ceph VM is not working for you, is to use the FakeS3 gem. This is included in the development gem group, so should have been installed by bundler.

Create a directory for the server to use to store files, and then start the server with:

$  bundle exec fakes3 -r /path/to/fakes3_root -p 8081 -H localhost

The settings should then be modified to the following:

S3:
  server: localhost:8081
  use_ssl: false
  access_key_id: anything
  secret_access_key: anything

Migrating from sqlite3 to mysql

This paragraph will explain the steps requires to migrate the database configuration from sqlite3 to mysql. The first step is to install mysql using homebrew.

$ brew install mysql

Once the installation is completed start your mysql server and log to it.

$ mysql.server start
$ mysql -u root -p

In the mysql console create the 3 environment databases required by rails.

mysql> create database dri_development;
mysql> grant all privileges on dri_development.* to 'root'@'localhost' identified by 'password';
mysql> flush privileges;

mysql> create database dri_test;
mysql> grant all privileges on dri_test.* to 'root'@'localhost' identified by 'password';
mysql> flush privileges;

mysql> create database dri_production;
mysql> grant all privileges on dri_production.* to 'root'@'localhost' identified by 'password';
mysql> flush privileges;

mysql> exit

The next step is to change the database configurations in the [yourProjectLocation]/config/database.yml as follows:

development:
  adapter: mysql
  database: dri_development
  username: root
  password: yourPWD
  host: localhost

test: &test
  adapter: mysql
  database: dri_test
  username: root
  password: yourPWD
  host: localhost

production:
  adapter: mysql
  database: dri_production
  username: root
  password: yourPWD
  host: localhost

cucumber:
  <<: *test

The last step is to install the mysql and mysql2 gems and to setup your new mysql databases by migrating and seeding them.

$ gem install mysql
$ gem install mysql2

$ bundle exec rake db:migrate
$ bundle exec rake db:seed
$ bundle exec rake db:test:prepare

Configuring Analytics

The application uses Google Analytics to store usage data.

All page views are send to Google Analytics using the google-analytics-rails gem. For published objects the object ID, parent collection ID and Depositing Organisation name are also sent.

When an object is viewed, or an asset file is downloaded the server also sends the view and download Events to Google Analytics via the Gabba gem.

The Analytics reports (/analytics) use the Legato gem to query the Google Analytics API.

In order for this to work a number of configuration changes are required at the Google Analytics side.

Configure a Google Analytics account for the site

Create a Google Analytics account for your site and get the Tracking ID number. This should be saved in the environments files as GA.tracker

Create Custom Dimensions

The following custom dimensions are required to be configured in the following order

* Dimension1 'collection'
* Dimension2 'organisation'
* Dimension3 'object'

This is done via the Admin Menu - Custom Definitions - Custom Dimensions. All three should have hit scope.

Create a project in the Google API console

Go to the Google API Console console.developers.google.com/ and view the Dashboard. You may be prompted to create a project. If not look for the projects list at the top of the screen beside the Google APIs logo. Click on this to see your project list and use the + button to create a new project.

Create credentials for the Google API project

You will need to create a service account credential. Click on the top menu and navigate to IAM & Admin - Service Accounts. Here create a new Service account giving it a role of Project - Viewer.

Save the account and download the key file in p12 format. Save this to the config dir and update the Settings.yml file with the following information:

profile_id

profile_id should be equal to the part after the p in the url for your analytics page. I.e. your analytics page will have an url of the form

analytics.google.com/analytics/web/#embed/report-home/a00000000w000000000p1234567890/

The part after the p is the profile id, in this case 1234567890.

issuer && sub

These should both be the service account ID which you will find in the Service Account section of the Google API Console console.developers.google.com/

keyfile

This should be the name of the p12 keyfile you downloaded from the Google API Console.

secret

This should match the secret that you were shown on downloading the p12 keyfile.

other

Other settings should be as follows

disable: false
token_credential_uri: 'https://accounts.google.com/o/oauth2/token'
audience: 'https://accounts.google.com/o/oauth2/token'
scope: 'https://www.googleapis.com/auth/analytics.readonly'