CALeDNA / Protecting Our River

Rails app the CALeDNA and Protecting Our River data portal.

Tech Stack

• Ruby 2.6
• Rails 5.2
• PostgreSQL + PostGIS
• Vue.js for pages that need dyanamic interactions
• Bootstrap 3.4 for styling
• Administrate for admin dashboard
• Devise for authenication
• Leaflet for maps
• Chart.js, D3.js, Vega for data visualizations
• Node.js 9 for installing javascript libraries
• Yarn javascript package manager
• Webpacker to bundle javascript
• [AWS]
• [Cloudfront]
• [Heroku]

Setup

Clone this repo.

Install libraries.

$ bundle install
$ yarn install

rake newb - runs setup.rb to fill out the environmental variables, run database migrations, and seed the database. Seeding the taxa tables (ncbi_names, ncbi_nodes) will take many minutes since the taxa tables have over 3.5 GB of data.

$ rake newb

Start server

Start rails server.

$ bin/rails s

This app uses Webpacker to handle javascript bundling. Optionally run webpack dev server to live reload javascript changes.

$ ./bin/webpack-dev-server

Optionally run sidekiq to handle background jobs. The most commonly used background jobs involve importing eDNA results.

$ redis-cli flushall
$ bundle exec sidekiq -q default -q mailer

Testing

To run the Rspec tests and Rubocop linter.

$ bin/rake

Data Sources

• KoBo API to import data from the Kobo webforms.
• NCBI for the eDNA taxonomy.
• GBIF API to display occurrence maps on taxa pages and GBIF downloaded dataset for additional occurrence data.
• GloBI for the taxa biological interactions.
• IUCN for the list of threatened / endangered species.

Notes

Protecting our River

• The code for the Protecting Our River site are in the project/pour branch. Code that can be used on both sites are in master. The code only used on PouR are in project/pour.
• To incorporate changes from the master branch to project/pour, you need switch to project/pour, git rebase master, and fix any conflicts.

Squarespace

• The UCeDNA.com site has a Squarespace site for the static content (ucedna.com), and a Rails app for the data portal (data.ucedna.com).
• The Rails app handles the registration and login. After users login to the Rails app, they are redirected to home page of Squarespace page. On the local development, the login redirects to localhost:9000, which is default port for the Squarespace development server.

Database

This app uses a lot of sql query and database types that only work on PostgreSQL.

This app uses multiple PostgreSQL database schemas.

• Most of the tables are in public.
• The tables that run the Pillar Point research project pages are in pillar_point.
• The tables used only for Protecting Our River are in pour.
• Tables containing data from GBIF, GloBI, iNaturalist, and 2017 NCBI taxonomy are in external.

See ./docs/fall_2020_database_schema.csv for an explanation of all the tables.

Update 3rd party data

bin/rake iunc_data:update_iunc_status - Connect to IUCN API to update the iucn_status in the ncbi_nodes table.

bin/rake wikidata:<task> - Query wikidata in order update the taxa links and images from external sites.

bin/rake gbif:<task> - Import occcurence and species that are downloaded from GBIF.