Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
OSF SciNet - Crowd sourcing academic citations
JavaScript Python CSS Shell
branch: master
Failed to load latest commit information.
docs
scinet Pyhonesque changes recommended by @jmcarp
test Adding groups for metric tracking of Citelet submissions in addition …
.gitignore Refactored testing/development config into one config. Adjusted views…
LICENSE Missing license
README.md Resolving Issue #44
requirements.txt Refactored for setup.py, moving tests.
setup.py Directory structure didn't match namespace, docs were broke, updated …

README.md

OSF SciNet -- Crowd-Sourcing the Scientific Citation Network

What is SciNet?

SciNet is a open source, collaborative project being developed at the Center for Open Science. The primary purpose of Scholarly is to provide the public with a free, open, and comprehensive dataset containing meta-data for academic citations as well as corresponding references. This dataset will allow the public to access, analyze, and distribute academic citation meta-data without restriction.

Relevant Wiki Pages:

Getting started with SciNet

Note: This set of packages is not production ready, but it should be robust enough for some early development. Additionally, these instructions were prepared using a Digial Ocean server running Ubuntu 12.04 but they can easily be adapted for running locally or on another cloud service.

  1. Create your droplet: http://digitalocean.com

  2. Check your email and make note of: IP Address of your droplet Username created for your droplet Password for your user

  3. SSH into your droplet:

    1. Open up a terminal and type:

      ssh <username>@<ip address>
      

      enter the password located in the email

  4. Set up a local (non-root) account:

    1. Create the user account:

      adduser <username>
      
    2. Give it sudo access:

      visudo
      # Below "root ALL=(ALL:ALL) All" enter
      <username> ALL=(ALL:ALL) ALL
      # Save the file and exit
      
  5. Logout of your current account and SSH into the server with the new one:

    1. From your terminal enter:

      exit
      ssh <username>@<ip address>
      
  6. Create a directory to hold our data:

    1. From your terminal enter:

      sudo mkdir /vol
      sudo chown <username>:<username> /vol
      cd /vol
      
  7. Install necessary packages (Note: This can take a minute):

    1. From your terminal enter:

      sudo apt-get update
      sudo apt-get install git python-flask python-pip build-essential python-dev python-requests
      sudo apt-get install python-pyquery uwsgi-plugin-python uwsgi nginx
      sudo pip install pymongo reppy nameparser uwsgi
      
  8. Clone and install SciNet:

    1. Checkout the repo:

      # Unless you have a forked SciNet use: https://github.com/centerforopenscience/scinet.git
      git clone https://github.com/user_name/repo.git
      
    2. Install required libraries:

      cd /vol/scinet
      sudo pip install -r requirements.txt
      

    Note: If you are installing on Ubuntu, remove the lxml requirement from requirements.txt before running pip install.

    1. Test it:

      cd /vol/scinet/scinet
      python main.py
      

    You should get a pymongo "Connection Refused" error.

  9. Install and setup mongodb:

    1. Add the 10gen repo:

      sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
      echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list
      sudo apt-get update
      sudo apt-get install mongodb-10gen
      
    2. Edit /etc/mongodb.conf and change dbpath=/var/lib/mongodb to dbpath=/vol/mongodb, and update permissions:

      sudo mkdir /vol/mongodb
      sudo chown -R mongodb /vol/mongodb
      
    3. Restart mongo:

      sudo service mongodb restart
      

      At this point, you should be able to (optionally) restart the instance and run:

      python /vol/scinet/scinet/main.py
      

      without errors.

  10. Get Citelet:

    1. Checkout the repo:

      cd /vol
      # Unless you have forked scinet-citelet use: https://github.com/centerforopenscience/scinet-citelet.git
      git clone https://github.com/user_name/repo.git
      
    2. Install required libraries::

      cd /vol/citelet
      sudo pip install -r requirements.txt
      
  11. Build Citelet:

    1. Verify config file is pointing to your desired urls:

      vi /vol/citelet/cfg.py
      

    For more information visit the Citelet docs.

    1. Run the fabric build process:

      fab rsudo deploy
      
    2. Test it:

      cd /vol/citelet/app
      python main.py
      
  12. Setup boto config so our keys will be loaded automatically:

    1. sudo vi /etc/boto.cfg:

      [Credentials]
      aws_access_key_id = <your access key>
      aws_secret_access_key = <your secret access key>
      
  13. Install and setup NGINX and uWSGI:

    1. Configure NGINX, for example, replace /etc/nginx/sites-available/default with:

      server {
          listen   80;
                server_name scinet;
            # scinet endpoint
              location /scinet {
                  uwsgi_pass unix:///tmp/scinet.sock;
              include uwsgi_params;
              # strip path before handing it to app
              uwsgi_param SCRIPT_NAME /scinet;
              uwsgi_modifier1 30;
          }
          # citelet endpoint
              location /citelet {
                  uwsgi_pass unix:///tmp/citelet.sock;
              include uwsgi_params;
              # strip path before handing it to app
              uwsgi_param SCRIPT_NAME /citelet;
              uwsgi_modifier1 30;
          }
      }
      
    2. Finally symlink it to its sites-enabled folder:

      sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default
      
    3. Configure UWSGI sockets, for example, create /etc/uwsgi/apps-available/scinet.ini and populate it with:

      [uwsgi]
      chdir = /vol/scinet/scinet
      uid = www-data
      gid = www-data
      chmod-socket = 666
      socket = /tmp/scinet.sock
      module = app
      callable = app
      
    4. And, create /etc/uwsgi/apps-available/citelet.ini and populate it with:

      [uwsgi]
      chdir = /vol/citelet/app
      uid = www-data
      gid = www-data
      chmod-socket = 666
      socket = /tmp/citelet.sock
      module = main
      callable = app
      
    5. Finally symlink them to their respective enabled folders:

      sudo ln -s /etc/uwsgi/apps-available/scinet.ini /etc/uwsgi/apps-enabled/scinet.ini
      sudo ln -s /etc/uwsgi/apps-available/citelet.ini /etc/uwsgi/apps-enabled/citelet.ini
      
    6. Enable the app and restart:

      sudo service nginx restart
      sudo service uwsgi restart
      

    The site should now be up and running. You can, for instance, install lynx and visit Test the site by visiting, for example, http:///scinet -or- http:///citelet.

  14. Still on the to-do list:

  • Pushing the site via git, with automatic server restarts, etc., implemented as git post commit hooks.
  • Automating the server setup process via the tools that Jeff and Lindsy were talking about.
Something went wrong with that request. Please try again.