Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-amplitude: Only fetch date_time_fields once per request #25315

Merged
merged 2 commits into from Apr 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -8967,7 +8967,7 @@
"sourceDefinitionId": "fa9f58c6-2d03-4237-aaa4-07d75e0c1396",
"name": "Amplitude",
"dockerRepository": "airbyte/source-amplitude",
"dockerImageTag": "0.2.1",
"dockerImageTag": "0.2.2",
"documentationUrl": "https://docs.airbyte.com/integrations/sources/amplitude",
"icon": "amplitude.svg",
"sourceType": "api",
Expand Down
Expand Up @@ -103,7 +103,7 @@
- name: Amplitude
sourceDefinitionId: fa9f58c6-2d03-4237-aaa4-07d75e0c1396
dockerRepository: airbyte/source-amplitude
dockerImageTag: 0.2.1
dockerImageTag: 0.2.2
documentationUrl: https://docs.airbyte.com/integrations/sources/amplitude
icon: amplitude.svg
sourceType: api
Expand Down
Expand Up @@ -1344,7 +1344,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-amplitude:0.2.1"
- dockerImage: "airbyte/source-amplitude:0.2.2"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/amplitude"
connectionSpecification:
Expand Down
Expand Up @@ -34,5 +34,5 @@ COPY source_amplitude ./source_amplitude
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.2.1
LABEL io.airbyte.version=0.2.2
LABEL io.airbyte.name=airbyte/source-amplitude
Expand Up @@ -68,6 +68,7 @@ class EventsExtractor(RecordExtractor):

def __post_init__(self, parameters: Mapping[str, Any]):
self.name = parameters.get("name")
self.date_time_fields = self._get_date_time_items_from_schema()

def _get_schema_root_properties(self):
schema_loader = JsonFileSchemaLoader(config=self.config, parameters={"name": self.name})
Expand All @@ -89,9 +90,8 @@ def _date_time_to_rfc3339(self, record: MutableMapping[str, Any]) -> MutableMapp
"""
Transform 'date-time' items to RFC3339 format
"""
date_time_fields = self._get_date_time_items_from_schema()
for item in record:
if item in date_time_fields and record[item]:
if item in self.date_time_fields and record[item]:
record[item] = pendulum.parse(record[item]).to_rfc3339_string()
return record

Expand Down
2 changes: 1 addition & 1 deletion connectors.md
Expand Up @@ -15,7 +15,7 @@
| **Amazon Ads** | <img alt="Amazon Ads icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/amazonads.svg" height="30" height="30"/> | Source | airbyte/source-amazon-ads:1.0.3 | generally_available | [docs](https://docs.airbyte.com/integrations/sources/amazon-ads) | [connectors/source/amazon-ads](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/amazon-ads) | [source-amazon-ads](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-ads) | <small>`c6b0a29e-1da9-4512-9002-7bfd0cba2246`</small> |
| **Amazon SQS** | <img alt="Amazon SQS icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/awssqs.svg" height="30" height="30"/> | Source | airbyte/source-amazon-sqs:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/amazon-sqs) | [connectors/source/amazon-sqs](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/amazon-sqs) | [source-amazon-sqs](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-sqs) | <small>`983fd355-6bf3-4709-91b5-37afa391eeb6`</small> |
| **Amazon Seller Partner** | <img alt="Amazon Seller Partner icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/amazonsellerpartner.svg" height="30" height="30"/> | Source | airbyte/source-amazon-seller-partner:1.0.1 | alpha | [docs](https://docs.airbyte.com/integrations/sources/amazon-seller-partner) | [connectors/source/amazon-seller-partner](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/amazon-seller-partner) | [source-amazon-seller-partner](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-seller-partner) | <small>`e55879a8-0ef8-4557-abcf-ab34c53ec460`</small> |
| **Amplitude** | <img alt="Amplitude icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/amplitude.svg" height="30" height="30"/> | Source | airbyte/source-amplitude:0.2.1 | generally_available | [docs](https://docs.airbyte.com/integrations/sources/amplitude) | [connectors/source/amplitude](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/amplitude) | [source-amplitude](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amplitude) | <small>`fa9f58c6-2d03-4237-aaa4-07d75e0c1396`</small> |
| **Amplitude** | <img alt="Amplitude icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/amplitude.svg" height="30" height="30"/> | Source | airbyte/source-amplitude:0.2.2 | generally_available | [docs](https://docs.airbyte.com/integrations/sources/amplitude) | [connectors/source/amplitude](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/amplitude) | [source-amplitude](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amplitude) | <small>`fa9f58c6-2d03-4237-aaa4-07d75e0c1396`</small> |
| **Apify Dataset** | <img alt="Apify Dataset icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/apify.svg" height="30" height="30"/> | Source | airbyte/source-apify-dataset:0.1.11 | alpha | [docs](https://docs.airbyte.com/integrations/sources/apify-dataset) | [connectors/source/apify-dataset](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/apify-dataset) | [source-apify-dataset](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-apify-dataset) | <small>`47f17145-fe20-4ef5-a548-e29b048adf84`</small> |
| **Appfollow** | <img alt="Appfollow icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/appfollow.svg" height="30" height="30"/> | Source | airbyte/source-appfollow:0.1.1 | alpha | [docs](https://docs.airbyte.com/integrations/sources/appfollow) | [connectors/source/appfollow](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/appfollow) | [source-appfollow](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-appfollow) | <small>`b4375641-e270-41d3-9c20-4f9cecad87a8`</small> |
| **Apple Search Ads** | <img alt="Apple Search Ads icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/apple.svg" height="30" height="30"/> | Source | airbyte/source-apple-search-ads:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/apple-search-ads) | [connectors/source/apple-search-ads](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/apple-search-ads) | [source-apple-search-ads](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-apple-search-ads) | <small>`e59c8416-c2fa-4bd3-9e95-52677ea281c1`</small> |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/amplitude.md
Expand Up @@ -43,6 +43,7 @@ The Amplitude connector ideally should gracefully handle Amplitude API limitatio

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------------------------------|
| 0.2.2 | 2023-04-19 | [25315](https://github.com/airbytehq/airbyte/pull/25315) | Refactor to only fetch date_time_fields once per request |
| 0.2.1 | 2023-02-03 | [25281](https://github.com/airbytehq/airbyte/pull/25281) | Reduce request_time_range to 4 hours |
| 0.2.0 | 2023-02-03 | [22362](https://github.com/airbytehq/airbyte/pull/22362) | Migrate to YAML |
| 0.1.24 | 2023-03-28 | [21022](https://github.com/airbytehq/airbyte/pull/21022) | Enable event stream time interval selection |
Expand Down