SDK based variants #1064
Merged
SDK based variants #1064
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pnadolny13
requested review from
sbalnojan,
afolson,
tayloramurphy,
rwfeather and
cjohnhanson
as code owners
✅ Deploy Preview for meltano-hub ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| api_key | True | None | The token to authenticate against the API service |
| start_date | False | None | The earliest record date to sync |
| end_date | False | None | The latest record date to sync |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-aftership --about
Version info
tap-aftership v0.0.4, Meltano SDK v0.13.1
Usage info
melty-bot % tap-aftership --help
Usage: tap-aftership [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-aftership",
"description": "AfterShip tap class.",
"version": "0.0.4",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"api_key": {
"type": [
"string"
],
"description": "The token to authenticate against the API service",
"secret": true,
"writeOnly": true
},
"start_date": {
"type": [
"string",
"null"
],
"format": "date-time",
"description": "The earliest record date to sync"
},
"end_date": {
"type": [
"string",
"null"
],
"format": "date-time",
"description": "The latest record date to sync"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"api_key"
]
}
}Discovered streams
trackings
|
Testing plugin
melty-bot % pipx install git+https://github.com/manuelapilongo/tap-aws-cognito.git |
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| user_token | True | None | Long-lived user access token with access to all pages. |
| pages | False | None | |
| start_date | False | 2022-10-01T00:00:00Z | Start date for fetching historical data. |
| insights_lookback_months | False | 24 | The lookback period for fetching insights data. Defaults to 12 months, meaning that we fetch insights only for the last 12 months and for posts published within the last 12 months. |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-facebook-pages --about
Version info
tap-facebook-pages v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % tap-facebook-pages --help
Usage: tap-facebook-pages [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-facebook-pages",
"description": "FacebookPages tap class.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"user_token": {
"type": [
"string"
],
"description": "Long-lived user access token with access to all pages.",
"secret": true,
"writeOnly": true
},
"pages": {
"type": [
"array",
"null"
],
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"string"
]
},
"access_token": {
"type": [
"string"
],
"secret": true,
"writeOnly": true
}
},
"required": [
"id",
"access_token"
]
}
},
"start_date": {
"type": [
"string",
"null"
],
"format": "date-time",
"default": "2022-10-01T00:00:00Z",
"description": "Start date for fetching historical data."
},
"insights_lookback_months": {
"type": [
"integer",
"null"
],
"default": 24,
"description": "The lookback period for fetching insights data. Defaults to 12 months, meaning that we fetch insights only for the last 12 months and for posts published within the last 12 months."
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"user_token"
]
}
}
|
Testing plugin
Usage infoDetected capabilities
|
|
Testing plugin
Usage infoDetected capabilities
|
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| tenant | True | None | The directory tenant that you want to request permission from. The value can be in GUID or a friendly name format. |
| client_id | True | None | The application ID that the Azure app registration portal assigned when you registered your app. |
| client_secret | True | None | The client secret that you generated for your app in the app registration portal. |
| stream_config | False | None | Custom configuration for streams. |
| api_version | False | v1.0 | The version of the Microsoft Graph API to use |
| auth_url | False | None | Override the Azure AD authentication base URL. Required if using a national cloud. |
| api_url | False | None | Override the Graph API service base URL. Required if using a national cloud. |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-ms-graph --about
Version info
tap-ms-graph v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % tap-ms-graph --help
Usage: tap-ms-graph [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-ms-graph",
"description": "MSGraph tap class.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"tenant": {
"type": [
"string"
],
"description": "The directory tenant that you want to request permission from. The value can be in GUID or a friendly name format."
},
"client_id": {
"type": [
"string"
],
"description": "The application ID that the Azure app registration portal assigned when you registered your app."
},
"client_secret": {
"type": [
"string"
],
"description": "The client secret that you generated for your app in the app registration portal.",
"secret": true,
"writeOnly": true
},
"stream_config": {
"type": [
"array",
"null"
],
"items": {
"type": "object",
"properties": {
"stream": {
"type": [
"string",
"null"
],
"description": "Name of stream to apply config."
},
"params": {
"type": [
"array",
"null"
],
"items": {
"type": "object",
"properties": {
"param": {
"type": [
"string",
"null"
],
"description": "Name of query parameter."
},
"value": {
"type": [
"string",
"null"
],
"description": "Value of query parameter."
}
}
}
}
}
},
"description": "Custom configuration for streams."
},
"api_version": {
"type": [
"string",
"null"
],
"default": "v1.0",
"description": "The version of the Microsoft Graph API to use"
},
"auth_url": {
"type": [
"string",
"null"
],
"description": "Override the Azure AD authentication base URL. Required if using a national cloud."
},
"api_url": {
"type": [
"string",
"null"
],
"description": "Override the Graph API service base URL. Required if using a national cloud."
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"tenant",
"client_id",
"client_secret"
]
}
}Discovered streams
users
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| access_token | True | None | Access token. |
| organization_id | True | None | The ID of an Organization. |
| limit | False | 100 | The response limit for paginated API streams. (Range: 0-100) |
| user_agent | False | None | The User agent to present to the API. |
| api_url | False | None | Override the url for the API service. |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-miro --about
Version info
tap-miro v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % tap-miro --help
Usage: tap-miro [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-miro",
"description": "Miro tap class.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"access_token": {
"type": [
"string"
],
"description": "Access token."
},
"organization_id": {
"type": [
"string"
],
"description": "The ID of an Organization."
},
"limit": {
"type": [
"integer",
"null"
],
"default": 100,
"description": "The response limit for paginated API streams. (Range: 0-100)"
},
"user_agent": {
"type": [
"string",
"null"
],
"description": "The User agent to present to the API."
},
"api_url": {
"type": [
"string",
"null"
],
"description": "Override the url for the API service."
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"access_token",
"organization_id"
]
}
}Discovered streams
organization members
|
Testing plugin
Usage infoDetected capabilities
|
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| client_id | True | None | |
| client_secret | True | None | The token to authenticate against the API service |
| start_date | False | None | %Y-%m-%d |
| end_date | False | None | %Y-%m-%d |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-paypal --about
Version info
tap-paypal v0.0.1, Meltano SDK v0.11.1
Usage info
melty-bot % tap-paypal --help
Usage: tap-paypal [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-paypal",
"description": "paypal tap class.",
"version": "0.0.1",
"sdk_version": "0.11.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"client_id": {
"type": [
"string"
]
},
"client_secret": {
"type": [
"string"
],
"description": "The token to authenticate against the API service"
},
"start_date": {
"type": [
"string",
"null"
],
"description": "%Y-%m-%d"
},
"end_date": {
"type": [
"string",
"null"
],
"description": "%Y-%m-%d"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"client_id",
"client_secret"
]
}
}Discovered streams
invoices
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| user | True | None | The user name used to connect to the Sybase database |
| password | True | None | The password for the connecting user to the Sybase database |
| host | True | None | The host name or IP Address running the Sybase Database |
| port | False | 2638 | The port that the database is running on, defaults to port 2638 |
| filter_dbs | False | None | To filter the discovery to a particular schema within a database. This is useful if you have a large number of schemas and wish to speed up the discovery. |
| use_date_datatype | False | 0 | To emit a date as a date without a time component or time without an UTC offset. This is helpful to avoid time conversions or to just work with a date datetype in the target database. If this boolean config item is not set, the default behaviour is false i.e. emit date datatypes as a datetime. It is recommended to set this on if you have time datetypes and are having issues uploading into into a target database. |
| tds_version | False | None | Set the version of TDS to use when communicating with Sybase Server (the default is None). This is used by pymssql with connecting and fetching data from Sybase databases. See the pymssql documentation and FreeTDS documentation for more details. |
| characterset | False | utf8 | The characterset for the database / source system. The default is utf8, however older databases might use a charactersets like cp1252 for the encoding. If you have errors with a UnicodeDecodeError: 'utf-8' codec can't decode byte .... then a solution is examine the characterset of the source database / system and make an appropriate substitution for utf8 like cp1252. |
| cursor_array_size | False | 1 | To make use of fetchmany(x) instead of fetchone(), use cursor_array_size with an integer value indicating the number of rows to pull. This can help in some architectures by pulling more rows into memory. The default if omitted is 1, the tap will still use fetchmany, but with an argument of 1, under the assumption that like cp1252. |
| use_singer_decimal | False | 0 | o emit all numeric values as strings and treat floats as string data types for the target, set use_singer_decimal to true. The resulting SCHEMA message will contain an attribute in additionalProperties containing the scale and precision of the discovered property |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-sybase --about
Version info
tap-sybase v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % tap-sybase --help
Usage: tap-sybase [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-sybase",
"description": "sybase tap class.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"user": {
"type": [
"string"
],
"description": "The user name used to connect to the Sybase database"
},
"password": {
"type": [
"string"
],
"description": "The password for the connecting user to the Sybase database",
"secret": true,
"writeOnly": true
},
"host": {
"type": [
"string"
],
"description": "The host name or IP Address running the Sybase Database"
},
"port": {
"type": [
"integer",
"null"
],
"default": 2638,
"description": "The port that the database is running on, defaults to port 2638"
},
"filter_dbs": {
"type": [
"string",
"null"
],
"description": "To filter the discovery to a particular schema within a database. This is useful if you have a large number of schemas and wish to speed up the discovery."
},
"use_date_datatype": {
"type": [
"boolean",
"null"
],
"default": false,
"description": "To emit a date as a date without a time component or time without an UTC offset. This is helpful to avoid time conversions or to just work with a date datetype in the target database. If this boolean config item is not set, the default behaviour is `false` i.e. emit date datatypes as a datetime. It is recommended to set this on if you have time datetypes and are having issues uploading into into a target database."
},
"tds_version": {
"type": [
"string",
"null"
],
"description": "Set the version of TDS to use when communicating with Sybase Server (the default is None). This is used by pymssql with connecting and fetching data from Sybase databases. See the pymssql documentation and FreeTDS documentation for more details."
},
"characterset": {
"type": [
"string",
"null"
],
"default": "utf8",
"description": "The characterset for the database / source system. The default is `utf8`, however older databases might use a charactersets like cp1252 for the encoding. If you have errors with a UnicodeDecodeError: 'utf-8' codec can't decode byte .... then a solution is examine the characterset of the source database / system and make an appropriate substitution for utf8 like cp1252."
},
"cursor_array_size": {
"type": [
"integer",
"null"
],
"default": 1,
"description": "To make use of fetchmany(x) instead of fetchone(), use cursor_array_size with an integer value indicating the number of rows to pull. This can help in some architectures by pulling more rows into memory. The default if omitted is 1, the tap will still use fetchmany, but with an argument of 1, under the assumption that like cp1252."
},
"use_singer_decimal": {
"type": [
"boolean",
"null"
],
"default": false,
"description": "o emit all numeric values as strings and treat floats as string data types for the target, set use_singer_decimal to true. The resulting SCHEMA message will contain an attribute in additionalProperties containing the scale and precision of the discovered property"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"user",
"password",
"host"
]
}
}
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| hostname | True | localhost | The Redis server hostname |
| port | True | 6379 | The Redis server port number |
| database | True | 0 | The Redis database number |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-redis --about
Version info
tap-redis v0.0.2, Meltano SDK v0.14.0
Usage info
melty-bot % tap-redis --help
Usage: tap-redis [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-redis",
"description": "Redis tap class.",
"version": "0.0.2",
"sdk_version": "0.14.0",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"hostname": {
"type": [
"string"
],
"default": "localhost",
"description": "The Redis server hostname"
},
"port": {
"type": [
"integer"
],
"default": 6379,
"description": "The Redis server port number"
},
"database": {
"type": [
"integer"
],
"default": 0,
"description": "The Redis database number"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"hostname",
"port",
"database"
]
}
}Discovered streams
recommendationmappingrecommendationtemplatemapping
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| app_id | True | None | The Tradera.com developer app ID |
| app_key | True | None | The Tradera.com developer app key |
| searches | False | None | Simple Search |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-tradera --about
Version info
tap-tradera v0.0.1, Meltano SDK v0.14.0
Usage info
melty-bot % tap-tradera --help
Usage: tap-tradera [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-tradera",
"description": "tradera tap class.",
"version": "0.0.1",
"sdk_version": "0.14.0",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"app_id": {
"type": [
"string"
],
"description": "The Tradera.com developer app ID",
"secret": true,
"writeOnly": true
},
"app_key": {
"type": [
"string"
],
"description": "The Tradera.com developer app key",
"secret": true,
"writeOnly": true
},
"searches": {
"type": [
"array",
"null"
],
"items": {
"type": "object",
"properties": {
"name": {
"type": [
"string"
],
"description": "Stream name"
},
"query": {
"type": [
"string"
],
"description": "Search query"
},
"category_id": {
"type": [
"integer",
"null"
],
"default": 0,
"description": "Category to search. Default 0 (all categories)"
},
"order_by": {
"type": [
"string",
"null"
],
"default": "EndDateAscending",
"enum": [
"Relevance",
"BidsAscending",
"BidsDescending",
"PriceAscending",
"PriceDescending",
"EndDateAscending",
"EndDateDescending"
]
}
},
"required": [
"name",
"query"
]
},
"description": "Simple Search"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"app_id",
"app_key"
]
}
}
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| client_id | True | None | The client identifier to authenticate against the API service. |
| client_secret | True | None | The client secret to authenticate against the API service. |
| start_date | True | None | The earliest datetime (UTC) to sync records. |
| auth_expiration | False | 300 | Expiraton in seconds for auth. (Range: 60-604800) |
| limit | False | 100 | The number of records to return per page. (Range: 0-100) |
| user_agent | False | None | User agent to present to the API. |
| api_url | False | None | Override the API service base URL. |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-podbean --about
Version info
tap-podbean v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % tap-podbean --help
Usage: tap-podbean [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-podbean",
"description": "Podbean tap class.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"client_id": {
"type": [
"string"
],
"description": "The client identifier to authenticate against the API service."
},
"client_secret": {
"type": [
"string"
],
"description": "The client secret to authenticate against the API service.",
"secret": true,
"writeOnly": true
},
"start_date": {
"type": [
"string"
],
"format": "date-time",
"description": "The earliest datetime (UTC) to sync records."
},
"auth_expiration": {
"type": [
"integer",
"null"
],
"default": 300,
"description": "Expiraton in seconds for auth. (Range: 60-604800)"
},
"limit": {
"type": [
"integer",
"null"
],
"default": 100,
"description": "The number of records to return per page. (Range: 0-100)"
},
"user_agent": {
"type": [
"string",
"null"
],
"description": "User agent to present to the API."
},
"api_url": {
"type": [
"string",
"null"
],
"description": "Override the API service base URL."
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
},
"required": [
"client_id",
"client_secret",
"start_date"
]
}
}Discovered streams
episodesnetwork_analytic_reportspodcast_analytic_reportspodcast_download_reportspodcast_engagement_reportspodcastsprivate_members
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| client_id | False | None | |
| client_secret | False | None | |
| refresh_token | False | None | |
| username | False | None | |
| password | False | None | |
| security_token | False | None | |
| is_sandbox | False | 0 | |
| action | False | update | |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: target-salesforce --about
Version info
target-salesforce v0.0.1, Meltano SDK v0.13.1
Usage info
melty-bot % target-salesforce --help
Usage: target-salesforce [OPTIONS]
Execute the Singer target.
Options:
--input FILENAME A path to read messages from instead of from
standard in.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ❌ 'discover'
- ❌ 'catalog'
- ❌ 'properties'
- ❌ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "target-salesforce",
"description": "Sample target for Salesforce.",
"version": "0.0.1",
"sdk_version": "0.13.1",
"capabilities": [
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"client_id": {
"type": [
"string",
"null"
]
},
"client_secret": {
"type": [
"string",
"null"
],
"secret": true,
"writeOnly": true
},
"refresh_token": {
"type": [
"string",
"null"
],
"secret": true,
"writeOnly": true
},
"username": {
"type": [
"string",
"null"
]
},
"password": {
"type": [
"string",
"null"
],
"secret": true,
"writeOnly": true
},
"security_token": {
"type": [
"string",
"null"
],
"secret": true,
"writeOnly": true
},
"is_sandbox": {
"type": [
"boolean",
"null"
],
"default": false
},
"action": {
"type": [
"string",
"null"
],
"default": "update",
"enum": [
"insert",
"update",
"delete",
"hard_delete"
]
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
}
}
}
|
Testing plugin
Auto-generated README.md
|
| Setting | Required | Default | Description |
|---|---|---|---|
| dialect | False | None | The Dialect of SQLAlchamey |
| driver_type | False | None | The Python Driver you will be using to connect to the SQL server |
| host | False | None | The FQDN of the Host serving out the SQL Instance |
| port | False | None | The port on which SQL awaiting connection |
| user | False | None | The User Account who has been granted access to the SQL Server |
| password | False | None | The Password for the User account |
| database | False | None | The Default database for this connection |
| sqlalchemy_eng_params | False | None | SQLAlchemy Engine Paramaters: fast_executemany, future |
| sqlalchemy_url_query | False | None | SQLAlchemy URL Query options: driver, TrustServerCertificate |
| batch_config | False | None | Optional Batch Message configuration |
| start_date | False | None | The earliest record date to sync |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
A full list of supported settings and capabilities is available by running: tap-mssql --about
Version info
tap-mssql v0.0.1, Meltano SDK v0.14.0
Usage info
melty-bot % tap-mssql --help
Usage: tap-mssql [OPTIONS]
Execute the Singer tap.
Options:
--state PATH Use a bookmarks file for incremental replication.
--catalog PATH Use a Singer catalog file with the tap.
--test TEXT Use --test to sync a single record for each
stream. Use --test=schema to test schema output
without syncing records.
--discover Run the tap in discovery mode.
--config TEXT Configuration file location or 'ENV' to use
environment variables.
--format [json|markdown] Specify output style for --about
--about Display package metadata and settings.
--version Display the package version.
--help Show this message and exit.
Detected capabilities
- ✅ 'discover'
- ✅ 'catalog'
- ✅ 'state'
- ✅ 'about'
JSON Metadata
{
"name": "tap-mssql",
"description": "mssql tap class.",
"version": "0.0.1",
"sdk_version": "0.14.0",
"capabilities": [
"catalog",
"state",
"discover",
"about",
"stream-maps",
"schema-flattening"
],
"settings": {
"type": "object",
"properties": {
"dialect": {
"type": [
"string",
"null"
],
"description": "The Dialect of SQLAlchamey"
},
"driver_type": {
"type": [
"string",
"null"
],
"description": "The Python Driver you will be using to connect to the SQL server"
},
"host": {
"type": [
"string",
"null"
],
"description": "The FQDN of the Host serving out the SQL Instance"
},
"port": {
"type": [
"integer",
"null"
],
"description": "The port on which SQL awaiting connection"
},
"user": {
"type": [
"string",
"null"
],
"description": "The User Account who has been granted access to the SQL Server"
},
"password": {
"type": [
"string",
"null"
],
"description": "The Password for the User account"
},
"database": {
"type": [
"string",
"null"
],
"description": "The Default database for this connection"
},
"sqlalchemy_eng_params": {
"type": [
"object",
"null"
],
"properties": {
"fast_executemany": {
"type": [
"string",
"null"
],
"description": "Fast Executemany Mode: True, False"
},
"future": {
"type": [
"string",
"null"
],
"description": "Run the engine in 2.0 mode: True, False"
}
},
"description": "SQLAlchemy Engine Paramaters: fast_executemany, future"
},
"sqlalchemy_url_query": {
"type": [
"object",
"null"
],
"properties": {
"driver": {
"type": [
"string",
"null"
],
"description": "The Driver to use when connection should match the Driver Type"
},
"TrustServerCertificate": {
"type": [
"string",
"null"
],
"description": "This is a Yes No option"
}
},
"description": "SQLAlchemy URL Query options: driver, TrustServerCertificate"
},
"batch_config": {
"type": [
"object",
"null"
],
"properties": {
"encoding": {
"type": [
"object",
"null"
],
"properties": {
"format": {
"type": [
"string",
"null"
],
"description": "Currently the only format is jsonl"
},
"compression": {
"type": [
"string",
"null"
],
"description": "Currently the only compression options is gzip"
}
}
},
"storage": {
"type": [
"object",
"null"
],
"properties": {
"root": {
"type": [
"string",
"null"
],
"description": "the directory you want batch messages to be placed in\nexample: file://test/batches"
},
"prefix": {
"type": [
"string",
"null"
],
"description": "What prefix you want your messages to have\nexample: test-batch-"
}
}
}
},
"description": "Optional Batch Message configuration"
},
"start_date": {
"type": [
"string",
"null"
],
"format": "date-time",
"description": "The earliest record date to sync"
},
"stream_maps": {
"type": [
"object",
"null"
],
"properties": {},
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
"stream_map_config": {
"type": [
"object",
"null"
],
"properties": {},
"description": "User-defined config values to be used within map expressions."
},
"flattening_enabled": {
"type": [
"boolean",
"null"
],
"description": "'True' to enable schema flattening and automatically expand nested properties."
},
"flattening_max_depth": {
"type": [
"integer",
"null"
],
"description": "The max depth to flatten schemas."
}
}
}
}
|
@pnadolny13 I'm able to find a few better images for some of the connectors - redis and aftership in particular. Also, if you're able to find stacked logos w/ wordmarks instead of the horizontal ones that'd be nice. |
tayloramurphy
approved these changes
Dec 5, 2022
|
Testing plugin
melty-bot % pipx install git+https://github.com/manuphatak/tap-codatio.git |
tayloramurphy
approved these changes
Dec 5, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds 21 new SDK based taps/targets continuing to burn down the list from #185.
New:
Alternative Variants:
Bugfix: The CI build preview was failing because some of these new plugins dont have metrics. Adding a check to make sure that metrics isnt null before using it looks to fix it.