SHARE is creating a free, open dataset of research (meta)data.
On the OSF
We'll be expanding this section in the near future, but, beyond using our API for your own purposes, harvesters are a great way to get started. You can find a few that we have in our list here.
Setup for testing
mkvirtualenv share -p `which python3.5` workon share
Once in the
share virtual environment, install the necessary requirements, then setup SHARE.
pip install -Ur requirements.txt python setup.py develop pyenv rehash # Only necessary when using pyenv to manage virtual environments
docker-compose assumes Docker is installed and running. Running
./bootstrap.sh will create and provision the database. If there are any SHARE containers running, make sure to stop them before bootstrapping using
docker-compose build web docker-compose run --rm web ./bootstrap.sh
Run the API server
# In docker docker-compose up -d web # Locally sharectl server
sharectl search setup
# In docker docker-compose up -d worker # Locally sharectl worker -B
Populate with data
This is particularly applicable to running ember-share, an interface for SHARE.
Harvest data from providers, for example
sharectl harvest com.nature sharectl harvest com.peerj.preprints # Harvests may be scheduled to run asynchronously using the schedule command sharectl schedule org.biorxiv.html # Some sources provide thousands of records per day # --limit can be used to set a maximum number of records to gather sharectl harvest org.crossref --limit 250
If the Celery worker is running, new data will automatically be indexed every couple minutes.
Alternatively, data may be explicitly indexed using
sharectl search # Forcefully re-index all data sharectl search --all
cd docs/ pip install -r requirements.txt make watch
Unit test suite