Harvey Needs API
Clone or download
Latest commit 00b8d9c Oct 26, 2017
Failed to load latest commit information.
app Merge branch 'master' into fix-bug-on-location-whitelist Oct 7, 2017
bin First commit, empty proj, with some light documentation Aug 31, 2017
config Translated to spanish Sep 22, 2017
db Flag user generated content as inappropriate Sep 21, 2017
lib Indexes and etags Sep 11, 2017
log First commit, empty proj, with some light documentation Aug 31, 2017
public Fix invalid parameter syntax Sep 21, 2017
test Administration screens for flagged markers Sep 21, 2017
tmp First commit, empty proj, with some light documentation Aug 31, 2017
vendor First commit, empty proj, with some light documentation Aug 31, 2017
.env.sample Add configuration and key for geocoding Sep 7, 2017
.gitignore First commit, empty proj, with some light documentation Aug 31, 2017
.ruby-version Specify ruby version Aug 31, 2017
.travis.yml compound index, feedback Sep 12, 2017
CODE_OF_CONDUCT.md Remove last vestiges of google sheet. Add conduct and reorg readme a bit Sep 4, 2017
Gemfile Create edit for pages and render markdown Sep 5, 2017
Gemfile.lock Create edit for pages and render markdown Sep 5, 2017
LICENSE Add software and content license Sep 4, 2017
Procfile Remove sidekiq since we are not on it Sep 2, 2017
README.md Update README.md Oct 9, 2017
Rakefile First commit, empty proj, with some light documentation Aug 31, 2017
config.ru First commit, empty proj, with some light documentation Aug 31, 2017
package.json First commit, empty proj, with some light documentation Aug 31, 2017


The Harvey Needs API

  • We serve JSON data here, open and fresh
  • We help client applications help those affeceted by Hurricane Harvey

Example Clients:

Developer Links



Speeding up results

You can use the If-Modified-Since header to check if there are any new results since that timestamp. If there are no new results, you'll receive a 304 NOT MODIFIED.

GET /api/v1/shelters HTTP/1.1
Host: api.harveyneeds.org
If-Modified-Since: Mon, 11 Sep 2017 21:23:03 -0000

The format for the date timestamp is rfc2822, though you can use any HTTP approved format.

  • With Moment.js: moment.utc().format("ddd, MM MMM YYYY HH:mm:ss [GMT]");
  • With Rails: Time.now.utc.rfc2822


Getting Started (Dev)


  • Ruby 2.4.1
  • Rails 5.1

User Administration

In rails console you'll want to create an admin user:

User.create! email: "youremail@example.com", password: "yourpassword", admin: true

Fork Repository and clone to local machine

Fork Repository
  • Prequisites
    • A github account and a repository into which to fork other repositories
  • Steps
    1. Navigate in a browser to https://github.com/sketch-city/harvey-api
    2. Click the Fork button in the upper right of the screenshot_create-service-account-key
    3. If prompted choose into where you want to form (e.g., if you are a member of multiple github repositories)
    4. You'll now have a forked version - your own version - of the repository.
    5. Go to "Clone or download" button on right of screen and copy that URL. It will be of the format git clone git@github.com:<YOUR OWN GITHUB REPOSITORY>/harvey-api.git
Clone Fork to local box
  • In the directory on your local box in which you plan to work run: git clone git@github.com:<YOUR OWN GITHUB REPOSITORY>/harvey-api.git

Setting up your .env file

  • GOOGLE_GEOCODER_API_KEY = Geocoding with Google Maps API, get one

  • GOOGLE_MAPS_JS_API_KEY = API key to show gmaps in views. See <script> at end of body in application.html.erb

    • Can be exactly the same as GOOGLE_GEOCODER_API_KEY

    • If you see error "Google Maps API error: ApiNotActivatedMapError" in your console, to activate:

      1. click above link to google developer console
      2. In left menu, click "Dashboard"
      3. In header, click "Enable APIs and Services"
      4. Search for "maps", select option for "Google Maps JavaScript API"
      5. In header, click "Enable"
  • AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY = Product Advertising API, see below

Note: this is optional; currently only needed to fetch new Amazon products from the Amazon Product Advertising API

You'll need to set the following ENV variables in a .env file

  1. Make a working copy of .env by runng this command at the terminal: cp .env.sample .env
  2. Get Amazon AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from Amazon's IAM. You'll need to create a PolicyName. You can name it "ProductAdvertisingAPI" with the following policy:
    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Action": "ProductAdvertisingAPI:*",
            "Resource": "*"

Creating a local database

  • Prerequisites

    • PostgreSQL is installed and running on your local machine
  • Method 1: Automatic

    • Run rails db:setup
  • Method 2: Manual

    • Create a postgres users It's recommended though optional that you use a distinct user for the harvey-api database
      • Example createuser harvey-api_development -P The -P flag will prompt you to create a password for the new user
    • Create the database (with the owner of the database set to user just created in the last step)
      • Example createdb -O harvey-api_development harvey-api_development
  • Run rails migrate to create schema

    • rails db:migrate
  • Import needs and shelters data from the production API:

    • rails api:import Sample output if successful
      Starting ImportSheltersJob 2017-09-03 18:33:03 +0000
      ImportSheltersJob Complete - {285}
      Starting ImportNeedsJob 2017-09-03 18:33:05 +0000
      ImportNeedsJob Complete - {92}
  • Test the API itself (Run API locally)

    • Example rails server Screenshot of Success: Screenshot

About the data import job

  • You can load Amazon Products by seeding your database: rails db:seed (or doing a full import rails amazon:import) The ActiveJobs and associated Rake task rails api:import, which imports data for shelters and needs from the production API into the application database, is intended for use in development and test environments only.

DO NOT RUN THIS JOB IN PRODUCTION. Since this job pulls data from the production API, running it in production can only be counter-productive, and would likely be destructive.

Development Process

Tests and Testing

Code should have tests, and any pull requests should be made only after you've made sure passes the test suite

Git and Github use

We force pull-requests from feature branches to master. Once something lands in master, it goes live instantly

Keeping your fork in sync
  • git remote add upstream git@github.com:sketch-city/harvey-api.git
  • git fetch upstream


Within your own forked repo create branches for each logical unit for work you do. One benefit of doing this is you'll be able to periodically sync your forked repo with upstream repo into the master branch without conflicting with work you may be doing.

Pull Requests

When you believe your code is ready to be merged into the upstream repository (sketch-city/harvey-api) by creating a pull request. Do this by

  • In github click the "Compare & pull request" button that github will present to you once you've committed changes to local repo
  • Describe what you changed and why; reference the issue(s) if any that your work addresses
More Information and Further Reading

Documentation Standards

Inline Comment Style

(Coming Soon)


Documentation such as READMEs (e.g., this document) are written in markdown per the [Github standard] (https://guides.github.com/features/mastering-markdown/)

Design Choices

  • Hosted on Heroku with PostGres
  • Uses ActiveJob (currently with sucker_punch)
  • MiniTest with Rails system tests

Thanks To:

Source Code Collaborators can be viewed: https://api.harveyneeds.org/contributors.html

But the API wouldn't mean anything without our volunteers:


Developer Resources

Errors you may get and what they mean

  • "PG::ConnectionBad" Looks like:
PG::ConnectionBad: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/tmp/.s.PGSQL.5432"?

Postgres is not running OR there is a connection problem to the database.


Software Code

This system's software code is licensed under the GPLv3.

Full license availabe in LICENSE

Data and Content

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.