The Research Software Directory is a content management system that is tailored to software.
The idea is that institutes for whom research software is an important output, can run their own instance of the Research Software Directory. The system is designed to be flexible enough to allow for different data sources, database schemas, and so on. By default, the Research Software Directory is set up to collect data from GitHub, Zenodo, Zotero, as well as Medium blogs.
For each software package, a product page can be created on the Research Software Directory if the software is deemed useful to others. Here is an example of what a product page may look like:
While the content shown on the product page can be completely customized, by default it includes a Mentions section, which can be used to characterize the context in which the software exists. The context may include links to scientific papers, but is certainly broader than that: for example, there may be links to web applications that demonstrate the use of the software, there may be links to videos on YouTube, tutorials on readthedocs.io or Jupyter notebooks, or there may be links to blog posts; really, anything that helps visitors decide if the software could be useful for them.
The Research Software Directory improves findability of software packages, partly because it provides metadata that helps search engines understand what the software is about, but more importantly because of the human centered text snippets that must be provided for each software package. After all, discovery of a software package is often not so much about finding it but knowing that you found it.
Try it out
Basically, these are the steps to get a copy of https://research-software.nl running locally (including data):
- Clone this repo
- Start the complete stack using
For details, see below.
Try it out, step 1/3: Clone this repo
Make sure to use the
--recursive flag, because this repository has
git clone --recursive https://github.com/research-software-directory/research-software-directory.git
Try it out, step 2/3: Configure
The research software directory is configured using a file with environment variables called
An example config file (
.env.example) is available, use it as a starting point.
cd research-software-directory cp .env.example .env
The config file has some place holder values (
changeme) they must be set by editing the
Below are instructions how to get the different tokens and keys.
- Set to GitHub organization name which users should be member of to login to admin site
- AUTH_GITHUB_CLIENT_ID + AUTH_GITHUB_CLIENT_SECRET
- Goto https://github.com/settings/developers -> OAuth Apps -> New OAuth App
- Set Authorization callback url: https://localhost/auth/get_jwt (replace localhost with the domain the site will be accessible on)
- Register application
- Use Client ID as value for AUTH_GITHUB_CLIENT_ID
- Use Client Secret as value for AUTH_GITHUB_CLIENT_SECRET
- Goto https://github.com/settings/tokens
- Generate new token
- Select no scopes
- Use token as value for GITHUB_ACCESS_TOKEN
- Create new private key
- Group permissions: Read Only
- Use api key as value for ZOTERO_API_KEY
- The Zotero group identifier, for example
1689348is the identifier for group https://www.zotero.org/groups/1689348/netherlands_escience_center
- The Zotero group identifier, for example
- Generate a random string (eg.
openssl rand -base64 32) and use as value for JWT_SECRET
- Generate a random string (eg.
Try it out, step 3/3: Start the complete stack using docker-compose
docker-compose --file docker-compose.yml --file docker-compose.prod.yml up --build
Wait (at maximum 5 minutes) until you see some output scroll by that is generated by the
tasks container, something like:
tasks_1 | 2018-07-11 10:30:02,990 cache_software [INFO] processing Xenon command line interface tasks_1 | 2018-07-11 10:30:03,013 cache_software [INFO] processing Xenon gRPC server tasks_1 | 2018-07-11 10:30:03,036 cache_software [INFO] processing xtas tasks_1 | 2018-07-11 10:30:03,059 cache_software [INFO] processing boatswain tasks_1 | 2018-07-11 10:30:03,080 cache_software [INFO] processing Research Software Directory tasks_1 | 2018-07-11 10:30:03,122 cache_software [INFO] processing cffconvert tasks_1 | 2018-07-11 10:30:03,149 cache_software [INFO] processing sv-callers
Open a web browser to verify that everything works as it should.
http://localhostshould show a local instance of the Research Software Directory
http://localhost/adminshould show the Admin interface to the local instance of the Research Software Directory
http://localhost/api/softwareshould show a JSON representation of all software in the local instance of the Research Software Directory
http://localhost/software/xenonshould show a product page (here: Xenon) in the local instance of the Research Software Directory
http://localhost/api/software/xenonshould show a JSON representation of a product (here: Xenon) in the local instance of the Research Software Directory
General workflow when making changes
Let's say you followed the steps above, and have a running instance of the Research Software Directory. Now you may want to make some changes to bring the frontend in line with your institute's branding. For example, you could follow the steps outlined here to change the fonts.
Now the question is, after making your changes, how do you get to see them? Here's how:
Go to the terminal where you started
Use Ctrl+C to stop the running instance of Research Software Directory
Check which docker containers you have with:
For example, mine says:
Name Command State Ports ---------------------------------------------------------------------------------------- researchsoftwaredirectory_admin_1 sh -c rm -rf /build/* && c ... Exit 0 researchsoftwaredirectory_auth_1 /bin/sh -c gunicorn --prel ... Exit 137 researchsoftwaredirectory_backend_1 /bin/sh -c gunicorn --prel ... Exit 137 researchsoftwaredirectory_certbot_1 sh /certbot.sh Exit 1 researchsoftwaredirectory_frontend_1 /bin/sh -c sh -c "mkdir -p ... Exit 137 researchsoftwaredirectory_mongo_1 /mongo.sh --bind_ip 0.0.0.0 Exit 137 researchsoftwaredirectory_nginx_1 /bin/sh -c sh /start.sh Exit 137 researchsoftwaredirectory_tasks_1 /bin/sh -c crond -d7 -f Exit 137
In the list above, container names consist of the directory name, the name of the service, plus a number. Use
docker-compose rmto delete the container by service name, e.g. the
docker-compose rm --force frontend docker rmi --force rsdnlesc/frontend
Rebuild containers as necessary, using:
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up --build
Frequently Asked Questions
Refer to the Frequently Asked Questions for more detailed answers to specific questions:
Making a release
Update the submodules to the latest version that the remote knows about
git submodule update --remote --merge # check the status git submodule status # if there are any SHAs preceded by a plus sign, you need # to git add them, e.g. git add db-dump git commit -m "updated db-dump submodule to the latest version"
Gather information on submodules such that it will make it to Zenodo, e.g.
git submodule status > submodules.sha.txt
Write the release notes, include the content of submodules.sha.txt
Generate the metadata file for Zenodo using cffconvert.
pip install --user cffconvert cffconvert --outputformat zenodo --ignore-suspect-keys --outfile .zenodo.json
# git add, commit, and push everything
Make sure that everything is pushed
cd $(mktemp -d) git clone --recursive https://github.com/research-software-directory/research-software-directory.git cd research-software-directory
Follow the notes from the 'For users' section above, and verify that it all works as it should.
Draft a new releasebutton here to make a release.