GWU's Sufia-based repository app for preservation and open access to scholarly output of the GWU community
Ruby HTML CSS Other
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
apache2_conf
app
bin
config
db
imagemagick_conf
lib
log
public
script
solr_conf
spec
test
tomcat_conf
vendor/assets
.gitignore
.rspec
.ruby-version
.travis.yml
Gemfile
Gemfile.lock
LICENSE
README.md
Rakefile
config.ru

README.md

GW ScholarSpace Build Status

=================

This is the (new) repository for the George Washington University Libraries' Sufia-based application. The repository for the Sufia 4 based app is located at https://github.com/gwu-libraries/gw-sufia/ .

Installation for Development

(For a recommended Production configuration, scroll down to Installation with Apache, Tomcat 7, and Passenger)

Dependencies

  • Install ubuntu package dependencies:

      % sudo apt-get update
      % sudo apt-get install git postgresql libpq-dev redis-server unzip openjdk-7-jre clamav-daemon curl imagemagick libapache2-mod-shib2 libapr1 libapr1-dev
    
  • Install RVM (for installation via SSL and other RVM installation information, refer to https://rvm.io/rvm/install)

      % curl -L https://get.rvm.io | bash -s stable
      % source ~/.rvm/scripts/rvm
    

    In the WRLC context, installing RVM via curl may require the following instead:

      % echo insecure >> ~/.curlrc
      % \curl -sSL --CAcert /etc/ssl/certs/ca-certificates.crt --CApath /etc/ssl/certs https://get.rvm.io | bash -s stable --ruby
      % source ~/.rvm/scripts/rvm
    
  • Install Ruby

      % rvm install ruby-2.2.1
    
  • Install Rails

      % gem install rails -v 4.2.6 -N
    

Install

  • Get the GW ScholarSpace code:

      % git clone https://github.com/gwu-libraries/scholarspace.git
    
  • Install gems

      % cd scholarspace
      % bundle install
    
  • Create a postgresql user

      % sudo su - postgres
      (postgres)% psql
      postgres=# create user YOURDBUSERNAME with createdb password 'YOURSFMDBPASSWORD';
    

    The result should be:

      CREATE ROLE
    
  • Create three databases (e.g. scholarspace_dev, scholarspace_test, scholarspace_prod)

      postgres=# \q
      (postgres)% createdb -O YOURDBUSERNAME YOURDEVDBNAME
      (postgres)% createdb -O YOURDBUSERNAME YOURTESTDBNAME
      (postgres)% createdb -O YOURDBUSERNAME YOURPRODDBNAME
      (postgres)% exit
    
  • Create the database.yml file

      % cd config
      % cp database.yml.template database.yml
    
      Edit database.yml to add your specific database names and credentials
    
  • Create the solr.yml file

      % cd config
      % cp solr.yml.template solr.yml
    
      Edit solr.yml to add your specific names and credentials
    
  • Create the fedora.yml file

      % cd config
      % cp fedora.yml.template fedora.yml
    
      Edit fedora.yml to add your specific Fedora repository names and credentials
    
  • Create the secure secret key. In production, put this in your environment, not in the file.

      % cd config
      % cp secrets.yml.template secrets.yml
      % rake secret
    
      Paste the secret key into the secrets.yml file (for dev and test)
    
  • Run the database migrations

      % rake db:migrate
    
  • Get a copy of hydra-jetty (if installing solr and fedora locally)

    (If jetty is already running on the same port (default 8983), it will need to be stopped using sudo service jetty stop)

    Hydra-Jetty requires Java 8. These Java 8 installation steps for Ubuntu Linux are recommended by ubuntu.com .

      % rake jetty:clean
      % rake sufia:jetty:config
      % rake jetty:start
    
  • Install fits.sh version 0.6.2 (check FITS for the latest 0.6.2 download)

      % cd /usr/local/bin
      % sudo curl http://projects.iq.harvard.edu/files/fits/files/fits-0.6.2.zip -o fits-0.6.2.zip
      % sudo unzip fits-0.6.2.zip
      % cd fits-0.6.2
      % sudo chmod a+x fits*.sh
    

Configure ImageMagick policies

  • Copy imagemagick_conf/policy.xml to /etc/ImageMagick (overwrite the default policy.xml)

Configure Contact form emailing

In order to enable the contact form page to send email when the user clicks Send, set the following properties in config/initializers/sufia.rb :

  • config.contact_email
  • config.from_email

Copy config/initializers/setup_mail.rb.template to config/initializers/setup_mail.rb . Set the SMTP credentials for the user as whom the app will send email.

Start a Redis RESQUE pool

    % cd scholarspace

Assign execute permissions to the script if it doesn't already have them:

    % sudo chmod +x script/restart_resque.sh

Run the script:

    % script/restart_resque.sh production

(or script/restart_resque.sh development if working in development mode)

Create the user roles

Run the rake task that creates user roles called admin and content-admin:

    % rake gwss:create_roles

At the rails console, add an initial user to the admin role. Make sure that your admin user has logged in at least once.

    % rails c
    > r = Role.find_by_name('admin')
    > r.users << User.find_by_user_key('YOUR_ADMIN_USER_EMAIL@gwu.edu')
    > r.save 

We will add the content-admin users shortly through the /roles UI.

(Optional) Populate the initial content blocks

Run the rake task that takes the content of the HTML files in config/locales/content_blocks and populates the associated content blocks. Note that for an existing instance, running this rake task will overwrite any chnages you've made to the content blocks!

    % rake gwss:populate_content_blocks

Run the application

    % rails s -p <PORT NUMBER> -b 0.0.0.0

(Optional) Add content-admin users

  • Ask each of the content-admin users to log in to the application at least once. Right now they will have read-only rights.

  • Log in as an admin user, and navigate to /roles

  • Select the content-admin role, and add each of the users to whom you wish to grant content-admin rights. These users should now be able to upload items and edit the items that they have uploaded (plus items transferred or proxied to them).

  • Note that removing users from roles through the /roles interface is currently broken, and must be accomplished through the rails console.

Installation with Apache, Tomcat 7, and Passenger

Note: Solr, Fedora, PostgreSQL and the GW ScholarSpace application can all be deployed on different servers if desired. If doing so, ensure that firewall ports are opened between the necessary servers and the GW ScholarSpace application server.

Dependencies

  • Install ubuntu 14.04 LTS package dependencies:

      % sudo apt-get update
      % sudo apt-get install git postgresql libpq-dev redis-server unzip clamav-daemon curl imagemagick libapache2-mod-shib2 tomcat7 libreoffice libcurl4-openssl-dev apache2-threaded-dev libapr1-dev libaprutil1-dev apache2-mpm-worker apache2-threaded-dev
    
  • Install Java 8 for 14.04 (on the Solr/Fedora server):

      % sudo apt-add-repository ppa:webupd8team/java
      % sudo apt-get update
      % sudo apt-get install oracle-java8-installer
      % sudo apt-get install oracle-java8-set-default
    
  • Install RVM for multi-users (for installation via SSL and other RVM installation information, refer to https://rvm.io/rvm/install) Installs RVM to /usr/local/rvm

    On the GW ScholarSpace server:

      % curl -L https://get.rvm.io | sudo bash -s stable
      % source ~/.rvm/scripts/rvm
      % rvm install ruby-2.2.1
      % sudo nano /etc/group
      
      Add users to the rvm group
    
  • (Optional) Set up Shibboleth integration on the GW ScholarSpace server:

    Please refer to https://github.com/gwu-libraries/shibboleth for the recommended steps for setting up the Shibboleth integration.

  • If Shibboleth has been setup on the GW ScholarSpace Server, enable Shibboleth in the appropriate environment file (ie: config/environments/production.rb):

    config.shibboleth = true

  • Install Rails on the GW ScholarSpace server:

      % gem install rails -v 4.2.6 -N
    
  • Create the directory structure

    On the GW ScholarSpace server:

      % mkdir /opt/scholarspace
      % mkdir /opt/install
      % mkdir /opt/xsendfile
    

    On the Solr/Fedora server (if deploying on a separate server - otherwise on the same server):

      % mkdir /opt/install
      % mkdir /opt/solr
      % mkdir /opt/fedora
      % sudo mkdir /var/log/solr
    
  • Create the necessary user groups and assign folder permissions

    On the GW ScholarSpace server:

      % sudo groupadd scholarspace
      
      Edit /etc/group and add users (including www-data) to the new group
      
      % sudo chown $USER:scholarspace /opt/scholarspace
      % sudo chown www-data:www-data /opt/xsendfile
    
  • Set up tomcat7 (on the Solr/Fedora server)

    Replace the /etc/default/tomcat7 file with the tomcat7 file from the /tomcat_conf folder in this repo. This will set solr home, fcrepo home, and java path.

    Set the owner of the /opt/fedora directory:

      % sudo chown tomcat7:tomcat7 /opt/fedora
    
  • Set up Solr (on the Solr server)

      % cd /opt/install
      % wget http://archive.apache.org/dist/lucene/solr/4.10.4/solr-4.10.4.tgz
      % tar xzvf solr-4.10.4.tgz
      % cd solr-4.10.4
      % sudo cp -R example/solr/* /opt/solr
      % sudo cp -R dist /opt/solr
      % sudo cp -R contrib /opt/solr
      % sudo cp -R /opt/solr/collection1 /opt/solr/development-core
      % sudo cp -R /opt/solr/collection1 /opt/solr/test-core
      % sudo chown -R tomcat7:tomcat7 /opt/solr
      % sudo chown tomcat7:tomcat7 /var/log/solr
    

    To configure Solr:

    • Edit /opt/solr/development-core/core.properties to set name=development-core

    • Edit /opt/solr/test-core/core.properties to set name=test-core

    • Replace the schema.xml and solrconfig.xml in /opt/solr/collection1/conf/ with the ones from the solr_conf folder in the repo.

    • Replace the solr.xml file in /opt/solr with the one from the solr_conf folder in the repo.

    Now to deploy Solr to run in Tomcat. From the /opt/install/solr-4.10.4 directory:

      % sudo cp dist/solr-4.10.4.war /var/lib/tomcat7/webapps/solr.war
      % sudo cp -R example/lib/ext/* /var/lib/tomcat7/webapps/solr/WEB-INF/lib
    

    Create the log4j.properties file in /var/lib/tomcat7/webapps/solr/WEB-INF/classes/ with the one from the tomcat_conf/solr folder in the repo.

    Set up log rotation by editing /etc/logrotate.d/solr and setting its contents to the following:

      /var/log/solr/solr.log { copytruncate daily rotate 5 compress missingok create 640 tomcat7 tomcat7 }
    
  • Set up fcrepo with audit support

      % cd /opt/install
      % wget https://github.com/fcrepo4-exts/fcrepo-webapp-plus/releases/download/fcrepo-webapp-plus-4.5.1/fcrepo-webapp-plus-audit-4.5.1.war
      % cp fcrepo-webapp-plus-audit-4.5.1.war /var/lib/tomcat7/webapps/fcrepo.war
    

    Replace the web.xml file in /var/lib/tomcat7/webapps/fcrepo/WEB-INF/web.xml with the one from the tomcat_conf/fcrepo-webapp folder in the repo

  • Ensure tomcat7 library files are (still) all owned by tomcat7

      % sudo chown -R tomcat7:tomcat7 /var/lib/tomcat7
    
  • Set up authentication to fcrepo

      % cd /etc/tomcat7
    

    Replace tomcat-users.xml with the tomcat-users.xml file from tomcat_conf folder in the repo.

    Edit tomcat-users.xml and replace the "dummypasswords" with your secure passwords.

  • Install postgresql on the Tomcat Server

      % sudo apt-get install postgresql
    
  • Create of backup of the existing Fedora instance (If one exists)

      % sudo mkdir /opt/fedora_backups
      % sudo chown -R tomcat7:tomcat7 /opt/fedora_backups
      % curl -X POST -u <FedoraUsername>:<FedoraPassword> --data "/opt/fedora_backups" yourserver.com/fcrepo/rest/fcr:backup
    

Verify that a backup was created in /opt/fedora_backups before proceeding

  • Create a postgresSQL database user for Fedora

      % sudo su - postgres
      (postgres)% psql
      postgres=# create user YOURDBUSERNAME with createdb password 'YOURDBPASSWORD';
    
  • Create a postgresSQL database for Fedora

      postgres=# \q
      (postgres)% createdb -O YOURDBUSERNAME ispn
      (postgres)% exit
    

Note the database name for Fedora must be 'ispn'

  • Create a Fedora settings folder

      % sudo mkdir /etc/fcrepo
      % sudo chown -R tomcat7:tomcat7 /etc/fcrepo
    
  • Copy the infinispan.xml file from the repo

      % cp infinispan.xml /etc/fcrepo/infinispan.xml
    

Edit this file with your database username and database password

  • Restart Tomcat

      % sudo service tomcat7 restart
    
  • Restore the Fedora Backup (If a previous Fedora repository was running on LevelDB)

      % curl -X POST -u <FedoraUsername>:<FedoraPassword> --data "/opt/fedora_backups" yourserver.com/fcrepo/rest/fcr:restore
    

Validate that the repository has been restored

  • Restart Tomcat7 Server

      % sudo service tomcat7 restart
    

Install the GW ScholarSpace app:

On the GW ScholarSpace server:

  • Get the GW ScholarSpace code:

      % git clone https://github.com/gwu-libraries/scholarspace.git
    
  • Install gems

      % cd scholarspace
      % bundle install --without development --deployment
    
  • Create a postgresql user

      % sudo su - postgres
      (postgres)% psql
      postgres=# create user YOURDBUSERNAME with createdb password 'YOURSFMDBPASSWORD';
    

    The result should be:

      CREATE ROLE
    
  • Create three databases (e.g. scholarspace_dev, scholarspace_test, scholarspace_prod)

      postgres=# \q
      (postgres)% createdb -O YOURDBUSERNAME YOURDEVDBNAME
      (postgres)% createdb -O YOURDBUSERNAME YOURTESTDBNAME
      (postgres)% createdb -O YOURDBUSERNAME YOURPRODDBNAME
      (postgres)% exit
    
  • Create the database.yml file

      % cd config
      % cp database.yml.template database.yml
    

    Edit database.yml to add your specific database names and credentials

  • Create the solr.yml file

      % cd config
      % cp solr.yml.template solr.yml
    

    Edit solr.yml to add your specific names and credentials

  • Create the blacklight.yml file

      % cd config
      % cp blacklight.yml.template blacklight.yml
    

    Edit blacklight.yml to add your specific names and credentials

  • Create the fedora.yml file

      % cd config
      % cp fedora.yml.template fedora.yml
    

    Edit fedora.yml to add your specific Fedora repository names and credentials

  • Create the secure secret key. In production, put this in your environment, not in the file.

      % cd config
      % cp secrets.yml.template secrets.yml
      % rake secret
    

    Paste the secret key into the secrets.yml file (for dev and test)

  • Run the database migrations

      % rake db:migrate RAILS_ENV=production
    
  • Download the hydra-jetty jar files. While we won't be running Solr and Fedora from hydra-jetty, we need to copy some of the jar files to the Solr deployment in order for full text indexing to work.

      % rake sufia:jetty:config
    
  • Move the extraction .jar files from hydra-jetty to your Solr instance (needed for full text indexing)

    If your Solr instance is on the same server as the GW ScholarSpace application, replace the /opt/solr/contrib/extraction folder with a copy of /opt/scholarspace/jetty/solr/lib/contrib/extraction:

      % rm -rf /opt/solr/contrib/extraction
      % cp -R jetty/solr/lib/contrib/extraction /opt/solr/contrib/
      % sudo service tomcat7 restart
    

    If your Solr instance is on a different server from the GW ScholarSpace application do the following:

    Download the /opt/scholarspace/jetty/solr/lib/contrib/extraction folder from the GW ScholarSpace application server via SFTP.

    On the Solr server, remove the /opt/solr/contrib/extraction directory that was installed with Solr 4.10.4:

      % rm -rf /opt/solr/contrib/extraction
    

    Upload the extraction folder from hydra-jetty to the server with your Solr instance into: /opt/solr/contrib/

      % sudo service tomcat7 restart
    
  • On the GW ScholarSpace server, install fits.sh version 0.6.2 (check FITS for the latest 0.6.2 download)

      % cd /usr/local/bin
      % sudo curl http://projects.iq.harvard.edu/files/fits/files/fits-0.6.2.zip -o fits-0.6.2.zip
      % sudo unzip fits-0.6.2.zip
      % cd fits-0.6.2
      % sudo chmod a+x fits*.sh
    

Configure ImageMagick policies

  • On the GW ScholarSpace server, copy imagemagick_conf/policy.xml to /etc/ImageMagick (overwrite the default policy.xml)

Configure the minter-state file path

  • Create a minter folder

    % sudo mkdir /opt/scholarspace/scholarspace_minter
    % sudo chown -R scholarspace_user:scholarspace_group /opt/scholarspace/scholarspace_minter
    
  • If an existing minter-state file exists in /tmp/minter-state copy it to the new folder

    % cp /tmp/minter-state /opt/scholarspace/scholarspace_minter/
    
  • Uncomment config.minter_statefile in config/initializers/sufia.rb

    # config.minter_statefile = '/opt/scholarspace/scholarspace_minter/minter-state'
    

Configure the tmp path

  • Create a tmp folder

    % sudo mkdir /opt/scholarspace/scholarspace_tmp
    % sudo chown -R scholarspace_user:scholarspace_group /opt/scholarspace/scholarspace_tmp
    
  • Uncomment config.temp_file_base in config/initializers/sufia.rb

    # config.temp_file_base = '/opt/scholarspace/scholarspace_tmp'
    

Configure Contact form emailing

In order to enable the contact form page to send email when the user clicks Send, set the following properties in config/initializers/sufia.rb :

  • config.contact_email
  • config.from_email

Copy config/initializers/setup_mail.rb.template to config/initializers/setup_mail.rb . Set the SMTP credentials for the user as whom the app will send email.

Start a Redis RESQUE pool

    % script/restart_resque.sh production

Create the user roles

Run the rake task that creates user roles called admin and content-admin:

    % rake gwss:create_roles RAILS_ENV=production

At the rails console, add an initial user to the admin role. Make sure that your admin user has logged in at least once.

    % rails c
    > r = Role.find_by_name('admin')
    > r.users << User.find_by_user_key('YOUR_ADMIN_USER_EMAIL@gwu.edu')
    > r.save 

We will add the content-admin users later through the /roles UI.

(Optional) Populate the initial content blocks

Run the rake task that takes the content of the HTML files in config/locales/content_blocks and populates the associated content blocks. Note that for an existing instance, running this rake task will overwrite any chnages you've made to the content blocks!

    % rake gwss:populate_content_blocks RAILS_ENV=production

Configure Passenger and Apache2

  • Set up Passenger

      % gem install passenger -v 5.0.19
      % passenger-install-apache2-module
      Select Ruby from the list of languages
    
  • Edit Apache for Passenger

    Copy the passenger.conf file from /opt/scholarspace/apache2_conf folder to /etc/apache2/conf-available/passenger.conf

  • Enable the passenger.conf file and the rewrite Apache mod

      % sudo a2enconf passenger.conf
      % sudo a2enmod rewrite
      % sudo service apache2 restart
    
  • Create and enable an Apache2 virtual host

    Copy the scholarspace.conf and scholarspace-ssl.conf files from the /opt/scholarspace/apache2_conf folder to /etc/apache2/sites-available/scholarspace.conf

    Enable modssl

      % sudo a2enmod ssl
    

    Generate certificates and place them in paths referenced in scholarspace-ssl.conf (modify the paths in scholarspace-ssl.conf if needed). Cert file names should also match.

      % sudo a2dissite 000-default.conf
      % sudo a2ensite scholarspace.conf
      % sudo a2ensite scholarspace-ssl.conf
    
  • Install mod_xsendfile (on the GW Scholarspace application server, if deploying on separate servers)

      % cd /opt/install
      % git clone https://github.com/nmaier/mod_xsendfile.git
      % cd mod_xsendfile
      % sudo apxs2 -cia mod_xsendfile.c
      % sudo service apache2 restart
    

Final Deployment

  • Prepare databases and assets for production

      % cd /opt/scholarspace
      % rake assets:precompile RAILS_ENV=production 
      % sudo service apache2 restart
    

(Optional) Add content-admin users

  • Ask each of the content-admin users to log in to the application at least once. Right now they will have read-only rights.

  • Log in as an admin user, and navigate to /roles

  • Select the content-admin role, and add each of the users to whom you wish to grant content-admin rights. These users should now be able to upload items and edit the items that they have uploaded (plus items transferred or proxied to them).

  • Note that removing users from roles through the /roles interface is currently broken, and must be accomplished through the rails console.

(Optional) Add Google Analytics

  • Enable Google Analytics in config/initializers/sufia.rb by editing the following lines:

       # Enable displaying usage statistics in the UI
       # Defaults to FALSE
       # Requires a Google Analytics id and OAuth2 keyfile.  See README for more info
       config.analytics = true
      
       # Specify a Google Analytics tracking ID to gather usage statistics
       config.google_analytics_id = 'UA-99999999-1'
    
       # Specify a date you wish to start collecting Google Analytic statistics for.
       config.analytic_start_date = DateTime.new(2015,11,10)
    
  • Copy the analytics.yml.template file in config

      % cp config/analytics.yml.template config/analytics.yml
    
  • Populate the analyitcs.yml file with your Google Analyitcs credentials. See: https://github.com/projecthydra/sufia#analytics-and-usage-statistics for setup details. Note that sufia seems to expect the .p12 file version of the private key, rather than the json version.

  • Set up a cron job to import GA stats nightly

    Test the script to make sure that it can run successfully. Make sure the script has execute permissions. Your analytics.yml file must also be set up correctly in order for the script to succeed.

      % cd /opt/scholarspace/script
    
      % sudo chmod +x import_stats.sh
    
      % ./import_stats.sh production
    

    If it runs successfully, proceed with adding the cron job:

      % crontab -e
    

    Add a line similar to the following:

      0 5 * * * /opt/scholarspace/script/import_stats.sh production
    

(Optional) Add SSL to Fedora and Solr Connections

These instructions are for redirecting port 8080 traffic on Tomcat to port 8443 and running SSL using the Apache Portable Runtime (APR).

  • Install Tomcat dependencies

      % sudo apt-get install libapr1 libapr1-dev libtcnative-1
    
  • Add the tomcat7 user to the ssl-cert group in /etc/group

     % sudo vi /etc/group
    
  • Generate your SSL certificates and key using the instructions provided here: https://github.com/gwu-libraries/ssl_howto

  • Copy the server_ssl.xml example from this repo to /etc/tomcat7/server.xml (note that you will need to overwrite server.xml (no ssl in the file name) with the contents of server_ssl.xml (ssl in the file name) from the repo.

     % sudo cp server_ssl.xml /etc/tomcat7/server.xml
    
  • Edit /etc/tomcat/server.xml and replace the dummy values for the following lines with your certificates and keys:

        SSLCertificateFile="/etc/ssl/certs/yourservername.cer"
        SSLCertificateChainFile="/etc/ssl/certs/yourservername.cer"
        SSLCertificateKeyFile="/etc/ssl/private/yourservername.pem"
    
  • Create a symbolic link to libtcnative1.so to address a Ubuntu/Tomcat bug

     % sudo ln -sv /usr/lib/x86_64-linux-gnu/libtcnative-1.so /usr/lib/
    
  • Replace the web.xml files for Solr and Fedora with the web_ssl.xml files from the repo:

    % cp tomcat_conf/fcrepo/web_ssl.xml /var/lib/tomcat7/webapps/fcrepo/WEB-INF/web.xml
    % cp tomcat_conf/solr/web_ssl.xml /var/lib/tomcat7/webapps/solr/WEB-INF/web.xml
    
  • Restart Tomcat and test access over HTTPS

     % sudo service tomcat7 restart