This is a Singer tap that produces JSON-formatted data following the Singer spec.
This tap:
- Pulls raw data from the PersistIQ v1. 0 API
- Extracts the following resources:
- Outputs the schema for each resource
- Incrementally pulls data based on the input state
- Endpoint: https://api.persistiq.com/v1/users
- Primary key fields: id
- Foreign key fields: none
- Replication strategy: FULL_TABLE
- Transformations: none
- Endpoint: https://api.persistiq.com/v1/leads
- Primary key fields: id
- Foreign key fields: none
- Replication strategy: FULL_TABLE
- Transformations: none
- Endpoint: https://api.persistiq.com/v1/campaigns
- Primary key fields: id
- Foreign key fields: lead_id > id, owner_id > id
- Replication strategy: FULL_TABLE
- Transformations: none
Clone this repository, and then install using setup. py. We recommend using a virtualenv:
> virtualenv -p python3 venv
> source venv/bin/activate
> python setup.py install
OR
> cd .../tap-persistiq
> pip install .
or alternatively:
> virtualenv -p python3 venv
> source venv/bin/activate
> pip install git+https://github.com/NickLeoMartin/tap-persistiq#egg=tap-persistiq
The following dependent libraries were installed.
> pip install singer-python
> pip install singer-tools
> pip install target-stitch
> pip install target-json
{
"api_key": "YOUR_PERSISTIQ_API_KEY",
"start_date": "2019-01-01T00:00:00Z",
"user_agent": "tap-persistiq <api_user_email@your_company.com>"
}
Optionally, also create a state.json
file. currently_syncing
is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off.
{
"currently_syncing": "leads",
"bookmarks": {
"users": "2019-09-27T22:34:39.000000Z",
"companies": "2019-09-28T18:23:53Z"
}
}
This creates a catalog. json for selecting objects/fields to integrate:
tap-persistiq --config config.json --discover > catalog.json
See the Singer docs on discovery mode here.
5. Run the Tap in Sync Mode (with catalog) and write out to state file
For Sync mode:
> tap-persistiq --config tap_config.json --catalog catalog.json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To load to json files to verify outputs:
> tap-persistiq --config tap_config.json --catalog catalog.json | target-json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To pseudo-load to Stitch Import API with dry run:
> tap-persistiq --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json