Recurly #1058
Recurly #1058
Conversation
| @@ -0,0 +1,4 @@ | |||
| { | |||
| "access_token": "<your access token>", | |||
There was a problem hiding this comment.
could you place this in the sample_configs directory and update it to match the spec.json file?
| ) -> Generator[AirbyteMessage, None, None]: | ||
| client = self._client(config_container) | ||
|
|
||
| catalog = AirbyteCatalog.parse_obj(self.read_config(catalog_path)) |
There was a problem hiding this comment.
this object is now a ConfiguredAirbyteCatalog
| catalog = AirbyteCatalog.parse_obj(self.read_config(catalog_path)) | |
| catalog = ConfiguredAirbyteCatalog.parse_obj(self.read_config(catalog_path)) |
There was a problem hiding this comment.
See #1065 for an example of the necessary change here
| "properties": { | ||
| "api_key": { | ||
| "type": "string", | ||
| "description": "Recurly API Key, use <a href=\"https://airbyte.recurly.com/integrations/api_keys\">admin</a> to generate this key." |
There was a problem hiding this comment.
current link points them to the Airbyte instance of recurly. Better to redirect to docs instead.
| "description": "Recurly API Key, use <a href=\"https://airbyte.recurly.com/integrations/api_keys\">admin</a> to generate this key." | |
| "description": "Recurly API Key. See the <a href=\"https://docs.airbyte.io/integrations/sources/recurly\">docs</a> for more information on how to generate this key." |
docs/integrations/sources/recurly.md
Outdated
|
|
||
| ### Output schema | ||
|
|
||
| Several output streams are available from this source (accounts, plans, invoices, transactions, etc.). |
There was a problem hiding this comment.
Could you:
- List the entities output (instead of "etc.")
- Add a URL to the recurly docs that helps the user understand what each of these entities mean
| Several output streams are available from this source (accounts, plans, invoices, transactions, etc.). | |
| Several output streams are available from this source: | |
| * accounts | |
| * .... | |
| return streams | ||
|
|
||
| def get_entities(self, entity_name): | ||
| if entity_name == "export_dates": |
There was a problem hiding this comment.
Could you add a comment explaining why there is a special handling of this stream?
| if resp.status_code == 200: | ||
| entity_data = resp.json()["data"] | ||
| while resp.json()["has_more"]: | ||
| resp = requests.get(f"{self.RECURLY_BASE_URL}{resp.json()['next']}", headers=self._headers) |
There was a problem hiding this comment.
is this correct? Recurly docs imply that the URL contained in next is complete and doesn't need to be appended to the base URL. To make sure that this is the case, is it possible to add a few hundred items to one of the streams just to future proof this? Ideally you can programmatically create 200 items in an entity via the API so it's not a ton of work on your end.
There was a problem hiding this comment.
That's correct, added 200+ coupons to test pagination.
| "Content-Type": "application/json", | ||
| } | ||
|
|
||
| def health_check(self): |
There was a problem hiding this comment.
Could you use return signatures on all methods e.g:
| def health_check(self): | |
| def health_check(self) -> Tuple[Boolean, object]: |
| self._client = RecurlyClient(api_key) | ||
| self._headers = { | ||
| "User-Agent": USER_AGENT, | ||
| "Authorization": "Basic %s" % b64encode(bytes(api_key + ":", "ascii")).decode("ascii"), |
There was a problem hiding this comment.
Why is it necessary to do this instead of just adding API key? Could you add a comment to the code so the context is not lost?
| PAGINATION = 100 | ||
|
|
||
| def __init__(self, api_key: str): | ||
| self.ENTITY_MAP = ["accounts", "coupons", "invoices", "plans", "measured_units", "subscriptions", "transactions", "export_dates"] |
There was a problem hiding this comment.
nit: If this is just a list then rename to ENTITIES
| @@ -0,0 +1,62 @@ | |||
| # Recurly Source | |||
There was a problem hiding this comment.
This readme looks out of date. Could you reference the latest template and update accordingly?
What
Add source connector for Recurly to load data.