Upgrade to Api v2 (#20)

* begin transition from harvest api v1 to v2

* update authentication method required for new v2 of the api
* add additional required config variables for tap usage
* update access_token request to comply with v2 of api
* update schemas to reflect data returned from the v2 api
* add additional schemas to reflect additional data points available

@see https://help.getharvest.com/api-v2/
@see #9

* generify `sync_endpoint` so that it is the only method that loads data and loops through data, thus removing redundancies

* allow each parent object loaded from the `sync_endpoint` to recursively sync additional endpoints (related to the parent object) through a `for_each_handler`
* remove api requests from the individual `sync_$item` methods, instead prefer using the generic `sync_endpoint` method

* store the id of nested objects instead of the full nested object

* automatically paginate all "collection" results (all calls to the `sync_endpoint` method)

* allow alternative key_properties, updated documentation, minor fixes

* allow key_properties to be set to anything other than just `["id"]` for use with pivot table style data
* update config.json and state.json documentation in README.md
* add additional schemas and subsequent loading-code that was missed in the first pass
* additional minor fixes such as
  * incorrect schema attribute names
  * missing attributes
  * incorrect schema attribute type/formats
* minor formatting tweaks

* skip syncing any endpoints that are not enabled by the given user's company, otherwise the user receives a 403 response error

* load company metadata from harvest api
* ensure specific features are either enabled or disabled before beginning endpoint syncs

* update module to v2.0.0, minor tweaks

* define default key_properties directly in `load_and_write_schema` method signature
* update version
* fix other minor inconsistencies

* determine account_id automatically as opposed to requiring account_id from user config

.gitignore

@@ -95,4 +95,5 @@ ENV/
 # Custom stuff
 env.sh
 config.json
+state.json
 .autoenv.zsh

README.md

@@ -2,7 +2,8 @@
 
 A singer.io tap for extracting data from the Harvest REST API, written in python 3.
 
-Author: Jordan Ryan (jordan@facetinteractive.com)
+API V1 Author: Jordan Ryan (jordan@facetinteractive.com)
+API V2 Author: Steven Hernandez (steven.hernandez@fostermade.co)
 
 ## Quick start
 
@@ -11,7 +12,7 @@ Author: Jordan Ryan (jordan@facetinteractive.com)
     Clone this repository, and then install using setup.py. We recommend using a virtualenv:
 
     ```bash
-    > virtualenv -p python 3 venv
+    > virtualenv -p python3 venv
     > source venv/bin/activate
     > python setup.py install
     ```
@@ -24,7 +25,7 @@ Author: Jordan Ryan (jordan@facetinteractive.com)
         "client_secret": "OAUTH_CLIENT_SECRET",
         "refresh_token": "YOUR_OAUTH_REFRESH_TOKEN",
         "start_date": "2017-04-19T13:37:30Z",
-        "account_name": "YOUR_ACCOUNT_NAME"
+        "user_agent": "MyApp (your.email@example.com)"
     }
     ```
 
@@ -34,18 +35,23 @@ Author: Jordan Ryan (jordan@facetinteractive.com)
     {
         "clients": "2000-01-01T00:00:00Z",
         "contacts": "2000-01-01T00:00:00Z",
-        "invoices": "2000-01-01T00:00:00Z",
+        "estimate_item_categories": "2000-01-01T00:00:00Z",
+        "estimate_messages": "2000-01-01T00:00:00Z",
+        "estimates": "2000-01-01T00:00:00Z",
+        "expense_categories": "2000-01-01T00:00:00Z",
+        "expenses": "2000-01-01T00:00:00Z",
         "invoice_item_categories": "2000-01-01T00:00:00Z",
-        "invoice_payments": "2000-01-01T00:00:00Z",
         "invoice_messages": "2000-01-01T00:00:00Z",
-        "expenses": "2000-01-01T00:00:00Z",
-        "expense_categories": "2000-01-01T00:00:00Z",
-        "projects": "2000-01-01T00:00:00Z",
+        "invoice_payments": "2000-01-01T00:00:00Z",
+        "invoices": "2000-01-01T00:00:00Z",
+        "project_tasks": "2000-01-01T00:00:00Z",
         "project_users": "2000-01-01T00:00:00Z",
+        "projects": "2000-01-01T00:00:00Z",
+        "roles": "2000-01-01T00:00:00Z",
         "tasks": "2000-01-01T00:00:00Z",
-        "project_tasks": "2000-01-01T00:00:00Z",
-        "people": "2000-01-01T00:00:00Z",
-        "time_entries": "2000-01-01T00:00:00Z"
+        "time_entries": "2000-01-01T00:00:00Z",
+        "user_projects": "2000-01-01T00:00:00Z",
+        "users": "2000-01-01T00:00:00Z"
     }
     ```
 

setup.py

@@ -3,7 +3,7 @@
 from setuptools import setup
 
 setup(name='tap-harvest',
-      version="1.1.1",
+      version="2.0.0",
       description='Singer.io tap for extracting data from the Harvest api',
       author='Facet Interactive',
       url='http://singer.io',
@@ -24,17 +24,30 @@
           'tap_harvest/schemas': [
               "clients.json",
               "contacts.json",
+              "estimate_item_categories.json",
+              "estimate_line_items.json",
+              "estimate_messages.json",
+              "estimate_message_recipients.json",
+              "estimates.json",
               "expense_categories.json",
               "expenses.json",
+              "external_reference.json",
               "invoice_item_categories.json",
-              "invoice_payments.json"
+              "invoice_messages.json",
+              "invoice_payments.json",
+              "invoice_message_recipients.json",
               "invoices.json",
-              "people.json",
               "project_tasks.json",
               "project_users.json",
               "projects.json",
+              "roles.json",
               "tasks.json",
               "time_entries.json",
+              "time_entry_external_reference.json",
+              "user_project_tasks.json",
+              "user_projects.json",
+              "user_roles.json",
+              "users.json",
           ],
       },
       include_package_data=True,