A singer.io tap for extracting data from the Harvest Forecast REST API, written in Python 3. Heavily derived from Jordan Ryan's Harvest Tap. Thanks for your work Jordan!
As the Harvest Forecast API is not yet public, this tap is experimental and liable to break at any time. Please keep this in mind if you run into any issues (and submit a PR if you can fix something broken).
Author: Robert Benjamin (@robertbenjamin)
-
Install
Clone this repository, and then install using setup.py. We recommend using a virtualenv:
virtualenv -p python3 venv source venv/bin/activate python setup.py install
-
Retrieve your oauth credentials from the Harvest Forecast API
Visit the developer tools page on Harvest's website and create a new oauth token
Paste the Client ID you got from the above page in the url of a browser like
https://id.getharvest.com/oauth2/authorize?client_id={OAUTH_CLIENT_ID}&response_type=code
. Now you're able to login, click 'authorize app' and then are redirected to a url like thishttps://id.getharvest.com/oauth2/authorize?code={OAUTH_REFRESH_TOKEN}&scope=all
. You will use thisOAUTH_REFRESH_TOKEN
in the following step to configure the oauth application -
Create your tap's
tap_config.json
file which should look like the following:{ "client_id": "OAUTH_CLIENT_ID", "client_secret": "OAUTH_CLIENT_SECRET", "refresh_token": "OAUTH_REFRESH_TOKEN", "start_date": "2017-04-19T13:37:30Z", "user_agent": "tap-harvest-forecast (your.email@example.com)" }
-
[Optional] Create the initial state file
{ "assignments": "2000-01-01T00:00:00Z", "clients": "2000-01-01T00:00:00Z", "milestones": "2000-01-01T00:00:00Z", "people": "2000-01-01T00:00:00Z", "projects": "2000-01-01T00:00:00Z" }
-
Setup the catalog
tap-harvest-forecast
can be run with:tap-harvest-forecast --config tap_config.json [--state state.json]
Run the tap in discovery mode to obtain the catalog:
tap-harvest-forecast --config tap_config.json --discover > catalog.json
You will need to add metadata in the catalog for stream/field selection, by adding
"selected": true
to themetadata
for each stream you wish to select intap_config.json
-
Run the application
Run the Tap in sync mode:
tap-harvest-forecast --config tap_config.json --catalog catalog.json
The output should consist of SCHEMA, RECORD, STATE, and METRIC messages. If you wish to test the tap with a target, see the documentation
If you see a completely blank run, like this:
This is caused by the catalog.json not containing "selected": "true"
in the metadata sections for each stream.
Just add that and you're good to go
Copyright © 2018 Stitch