RIS2WEB: Publishing a bibliography on the web

This project represents an effort to use a Zotero group library as the basis of creating a bibliography suitable for publishing on the web. The citations themselves express linked data through the http://schema.org vocabulary. URIs for authors are added to the database after the Zotero library has been processed, as Zotero itself offers no linked data support.

To improve visibility of the citations in general search engines, we also generate a sitemap and robots.txt file that advertises the sitemap.

Although any RIS file could be used as a source for the bibliography, the ris2sql processor assumes some quirks of Zotero (such as Editor = 'A3' and Series Editor = 'A2').

Licences

The code is licensed under the GNU General Public License (GPL) 3.

The data is licensed under the Open Data Commons - Attribution (ODC-BY) license.

Installing prerequisites on Ubuntu 14.04

sudo apt-get install git postgresql python3 python3-flask python3-pip nginx
sudo pip install psycopg2 flask-babelex

Installing prerequisites on Fedora 29

First we get the web server and database server up and running:

sudo dnf install nginx postgresql-server
sudo postgresql-setup --initdb --unit postgresql
sudo systemctl start nginx
sudo systemctl start postgresql

The we install the Python dependencies, and use virtualenv to avoid polluting the system site packages:

sudo dnf install pipenv
pipenv install
pipenv shell

Getting up and running

The following instructions should more or less enable you to get a clone of this instance up and running on the default Flask port:

sudo su -c 'createuser -s <username>' postgres
sudo su -c 'createdb bibliography' postgres
git clone https://github.com/dbs/ris2web.git
cd ris2web
./create_bibdb && ./sitemap_maker
pybabel compile -d translations/
./ris2web

Once ris2web is running, you should be able to access the site on the same machine using a browser pointed at http://localhost:5000.

Translating

Pybabel is the utility used to pull out translateable text from templates/py files. Simply extract and update to put in new translations.

pybabel extract -F babel.cfg -o messages.pot .
pybabel update -i messages.pot -d translations
pybabel compile -d translations

A note about committing translations

When adding new messages to messages.pot or new translations to messages.po, ALWAYS make sure to only partially stage the actual messages added. Pybabel updates a header with the date of the last message extract, so it will cause a merge conflict if it’s staged into the commit.

Updating the database

1. Back up the database with a command like pg_dump --no-owner --clean --dbname bibliography > bibliography.sql.

2. Update the value of stop_date in config.ini to the date of the last update.

3. Start a pipenv shell: pipenv shell

4. Run the update: ./ris2web_api

5. Start the web service and test out the changes: ./ris2web

Credits

• Bibliography compilation and overall direction by Desmond Maley.

• Faculty participation by Reuben Roth, Laurentian University.

• Bibliography compilation by Fiona McQuade-Crangle, with contributions from Carley Whittle, both from Laurentian University.

• Code by Dan Scott, Laurentian University.